DETAILED ACTION
This is in response to applicant’s amendment/response filed on 02/17/2026, which has been entered and made of record. Claims 1, 6, 9 have been amended. Claims 1-9 are pending in the application. The objections to claims 6, 9 is/are withdrawn in view of amendments. The rejections under 35 USC § 112 to claims 1-9 are withdrawn.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 6, 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yano et al. (US 20110210868) in view of MURAKAMI (US 20250252646).
Regarding claim 1, Yano discloses A method for superimposing and displaying of extra-information on image data displayed on an image display unit of a digital vision system, the vision system comprises an image capturing unit having an image sensor, an image processing unit and an image display unit having and image display area (Yano, “[0036] A parking assistance apparatus according to an embodiment is connected, via a network, to at least one camera mounted on a vehicle and a display apparatus that may display an image, and assists with parking the vehicle onto a target parking position. The parking assistance apparatus superimposes, in the same coordinate system as the overhead-view image generated, an image of the vehicle on the overhead-view image and a parking space figure at a position a predetermined distance away from the vehicle on the overhead-view image, and outputs the resulting overhead-view image. [0049] FIG. 1 is a schematic diagram of the parking assistance apparatus 3, and the function units of the parking assistance apparatus 3 may be realized by execution of a program read by a central processing unit (CPU)”), a sensor coordinate system is assigned to the image sensor of the image capturing unit, the sensor coordinate system has a known position and orientation with respect to the vehicle, the sensor coordinate system has an origin of the sensor coordinate system, an extra-information coordinate system is assigned to the extra-information, and the extra-information coordinate system has an origin of the extra-information coordinate system (Yano, “[0007] a parking assistance apparatus includes: a generation unit that generates an overhead-view image as seen from a predetermined viewpoint, in accordance with an image captured by at least one image capturing apparatus mounted in a vehicle. [0067] In the following, in the overhead-view image of FIG. 6B, description will be made by treating the upper left corner as the origin 0. [0109] In the following, in an overhead-view image of FIG. 15A, description will be made by treating the upper left corner as the origin 0. [0069] Thus, by using the actual, horizontal width Dx and vertical width Dy of an area displayed by the overhead-view image, the actual width W and length H of the vehicle 30, the reference-point coordinates (X, Y) of the vehicle 30 in an overhead coordinate system are obtained. Claim 1, a generation unit that generates an overhead-view image as seen from a predetermined viewpoint, in accordance with an image captured by at least one image capturing apparatus mounted in a vehicle; and an output unit that superimposes, in a same coordinate system as the overhead-view image generated by the generation unit”), the method comprising:
extracting image data from an extraction area of the image sensor (Yano, fig.6B, “[0042] If the number of cameras is one, a 360-degree camera may be used to capture an image of the surroundings of the vehicle 30. [0043] The generation unit 3a of the parking assistance apparatus 3 performs processing for generating (synthesizing) an overhead-view image as seen from a predetermined viewpoint in accordance with images that have been transmitted from the image capturing apparatus 5”);
superimposing the extra-information on the extracted image data in the image processing unit; and displaying the image data together with the overlayed extra-information on the image display area of the image display unit (Yano, “[0043] The output unit 3b of the parking assistance apparatus 3 superimposes an image of the vehicle 30 and a parking space figure illustrating a target parking position on the overhead-view image in a same coordinate system as the overhead-view image, and performs processing for outputting the resulting image to the display apparatus 7”),
wherein the position of the sensor extraction area in the sensor coordinate system is shiftable and/or the size of the sensor extraction area is variable (Yano, “[0080] For example, an overhead-view image as illustrated in FIG. 7A is displayed on the display 21 by the above-described processing (operations S401 to S413 in FIG. 4). Since this overhead-view image is updated at predetermined intervals, the overhead-view image changes as the vehicle 30 moves”),
wherein the extra-information is superimposed on the extracted image data such that the origin of the extra-information coordinate system is assigned to a fixed point in the sensor coordinate system (Yano, “[0043] The output unit 3b of the parking assistance apparatus 3 superimposes an image of the vehicle 30 and a parking space figure illustrating a target parking position on the overhead-view image in a same coordinate system as the overhead-view image, and performs processing for outputting the resulting image to the display apparatus 7”. Therefore, the image of the vehicle is superimposed on the overhear-view image such that the origin of the image of the vehicle is assigned to the same (a fixed point) as the camera coordinate system), and
wherein the orientation of the extra-information coordinate system is fixed with respect to the sensor coordinate system, independent of a shifting of the position of the sensor extraction area and/or a change of size of the sensor extraction area (Yano, “[0080] For example, an overhead-view image as illustrated in FIG. 7A is displayed on the display 21 by the above-described processing (operations S401 to S413 in FIG. 4). Since this overhead-view image is updated at predetermined intervals, the overhead-view image changes as the vehicle 30 moves. However, the vehicle 30 is always displayed at the center of the overhead-view image.”).
On the other hand, Yano fails to explicitly disclose but MURAKAMI discloses A method for superimposing and displaying of extra-information on image data displayed on an image display unit of a digital vision system for a vehicle (MURAKAMI, “[0218] Conceivable examples of the display device include, but are not limited to a mobile phone…AR glasses, AR goggles, a monitor capable of displaying AR, an device connected to a monitor capable of displaying AR, VR glasses, VR goggles, a monitor capable of displaying VR, an device connected to a monitor capable of displaying VR, mixed reality (MR) glasses, a monitor capable of displaying MR, an device connected to a monitor capable of displaying MR, a car navigation system, a head mounted display, an device connected to a head mounted display, a monitor, an device connected to a monitor, a projector, an device connected to a projector, etc.”);
a sensor coordinate system is assigned to the image sensor, the sensor coordinate system has a scaling of the sensor coordinate system, an extra-information coordinate system is assigned to the extra-information, and the extra-information coordinate system has a scaling of the extra-information coordinate system; the ratio of the scaling of the extra-information coordinate system to the scaling of the sensor coordinate system is affixed a ratio m:1 (MURAKAMI, “[0638] “real space #X” and “virtual space #Y” (where X and Y are assumed to be integers greater than or equal to 0) may be in a digital twin relationship. However, “real space #X” and “virtual space #Y” need not be in a complete digital twin relationship, and for example, “real space #X” and “virtual space #Y” may partially be in a digital twin relationship. Note that the size of the space represented by “real space #X” and “virtual space #Y” may be the same or may be different. When the size of the space represented by real space #X and virtual space #Y is different, the scale ratio between real space #X and virtual space #Y may be different for each axis that defines the space, or it may be the same. Here, the axes that define the space are the three axes of a three-dimensional coordinate system in the case of a three-dimensional space, and the two axes of a two-dimensional coordinate system in the case of a two-dimensional space”. Therefore, when the virtual space and real space are in a digital twin relationship, the ratio of scaling is affixed to 1:1).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined MURAKAMI and Yano, to include all limitations of claim 1. That is, applying the same scale ratio between the virtual space and real space in the display of a car navigation system of MURAKAMI to the display system independent of a shifting of the position of the sensor extraction area of Yano. The motivation/ suggestion would have been to make a character (avatar) appear in a virtual space based on the position of a terminal (or user) in real space (MURAKAMI, [0012]).
Regarding claim(s) 9, all limitation of claim 9 are covered by claim 1, therefore, it is interpreted and rejected for the similar reasons set forth in claim(s) 1.
Regarding claim 2, Yano in view of MURAKAMI discloses The method according to claim 1.
On the other hand, Yano fails to explicitly disclose but MURAKAMI discloses wherein the sensor coordinate system and the extra-information coordinate system are cartesian coordinate systems having an x-axis and an y-axis, respectively (MURAKAMI, “[0488] For example, consider a three-dimensional space as illustrated in FIG. 16C. Accordingly, there is an x-axis, a y-axis, and a z-axis. [0638] the scale ratio between real space #X and virtual space #Y may be different for each axis that defines the space, or it may be the same. Here, the axes that define the space are the three axes of a three-dimensional coordinate system in the case of a three-dimensional space, and the two axes of a two-dimensional coordinate system in the case of a two-dimensional space”), and
the ratio of the scaling of the x-axis of the extra-information coordinate system to the scaling of the x-axis of the sensor coordinate system is a fixed ration m1:1, and the ratio of the scaling of the y-axis of the extra-information coordinate system to the scaling of the y-axis of the sensor coordinate system is a fixed ration m2:1 (MURAKAMI, “ [0638] “real space #X” and “virtual space #Y” (where X and Y are assumed to be integers greater than or equal to 0) may be in a digital twin relationship. However, “real space #X” and “virtual space #Y” need not be in a complete digital twin relationship, and for example, “real space #X” and “virtual space #Y” may partially be in a digital twin relationship. Note that the size of the space represented by “real space #X” and “virtual space #Y” may be the same or may be different. When the size of the space represented by real space #X and virtual space #Y is different, the scale ratio between real space #X and virtual space #Y may be different for each axis that defines the space, or it may be the same. Here, the axes that define the space are the three axes of a three-dimensional coordinate system in the case of a three-dimensional space, and the two axes of a two-dimensional coordinate system in the case of a two-dimensional space”. Therefore, when the virtual space and real space are in a digital twin relationship, the ratio of scaling in x-axis and y-axis are both affixed to 1:1). The same motivation of claim 1 applies here.
Regarding claim 6, Yano in view of MURAKAMI discloses The method according to claim 1.
Yano further discloses changing the position and/or size of the sensor extraction area (Yano, “[0080] For example, an overhead-view image as illustrated in FIG. 7A is displayed on the display 21 by the above-described processing (operations S401 to S413 in FIG. 4). Since this overhead-view image is updated at predetermined intervals, the overhead-view image changes as the vehicle 30 moves. [0085] Thus, as illustrated in FIGS. 7D to 7H, the overhead-view image of the surroundings of the vehicle 30 is updated as the vehicle 30 moves”).
Claim(s) 3, 4, 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yano et al. (US 20110210868) in view of MURAKAMI (US 20250252646), and further in view of Morita et al. (US 20070132662).
Regarding claim 3, Yano in view of MURAKAMI discloses The method according to claim 1.
On the other hand, Yano in view of MURAKAMI fails to explicitly disclose but Morita discloses wherein the image capturing unit and/or the image sensor are mechanically adjustable, such that an adjustment directional movement component results, which corresponds to a shift in the sensor coordinate system (Morita, “[0050] FIG. 11 is a view showing an example of an image which is displayed on the display screen of the HMD 110 when the position and orientation of the viewpoint of an observer 100 are moved. [0098] The position and orientation of the video camera 111 in the sensor coordinate system will be handled as the position and orientation of the viewpoint of the observer 100 hereinafter”),
wherein upon a mechanical adjustment of the image capturing unit and/or the image sensor having an adjustment directional movement component corresponding to a shift in the sensor coordinate system and by a movement distance, the extra-information is superimposed such on the extracted image data that the position of the origin of the extra-information coordinate system is shifted corresponds to the movement distance in the sensor coordinate system with respect to the fixed point in the sensor coordinate system (Morita, “[0100] The coordinate system in the virtual space shares the "origin and three axes" with the sensor coordinate system. That is, the sensor coordinate system matches the coordinate system of the virtual space. Hence, the CPU 3101 can generate the image of the virtual space seen from the position and orientation of the viewpoint of the observer 100 by using the data of the position and orientation of the viewpoint of the observer 100 and the data of the virtual space loaded from the external storage device 3106 to the RAM 3102 in advance. [0103] The generated virtual space image is superimposed on the physical space image stored in the RAM 3102 in advance. The superimposed image is output onto the display screen of the HMD 110 through the I/F 3107 as the mixed reality space image”. Therefore, the origin of the virtual space coordinate system is shifted corresponding the movement in the sensor coordinate system with respect to the origin in the sensor coordinate system).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Morita into the combination of MURAKAMI and Yano, to include all limitations of claim 3. That is, adding the corresponding origin shift of the overlaid image of Morita to the vehicle display system of MURAKAMI and Yano. The motivation/ suggestion would have been when a virtual object including at least one part and superimposed on a physical space is presented to an observer, displaying information about the parts without damaging the sense of depth with respect to anther image (Morita, [0008]).
Regarding claim 4, Yano in view of MURAKAMI discloses The method according to claim 1.
On the other hand, Yano in view of MURAKAMI fails to explicitly disclose but Morita discloses wherein the extra-information is stored in the image processing unit as a fixed defined graphics (Morita, “[0091] Data stored in the external storage device 3106 includes a data group to generate a list image (to be described later) and data to generate an image of each virtual object (virtual object of copying machine in FIG. 1) which forms the virtual space. [0108] Displayed information contains the material names and texture names of the parts (parts corresponding to the children of the console portion 500: liquid crystal display panel, online button, discharge button, up button, down button, right button, and left button in FIG. 6) included in the console portion 500 in FIG. 6. However, the present invention is not limited to this, and any other information may be displayed. [0142] the display position of a list image 701 does not change according to the processing in step S1912”). The same motivation of claim 3 applies here.
Regarding claim 8, Yano in view of MURAKAMI discloses The method according to claim 1.
On the other hand, Yano in view of MURAKAMI fails to explicitly disclose but Morita discloses selecting a partial quantity of the extra-information data for display on the image display unit (Morita, “[0198] FIG. 26 is a view showing a display example of the list image of a portion pointed by gesture. When the stylus 120 is moved to enclose the virtual part corresponding to the node to be selected, the node to display details is selected. In this example, the upper paper cassette is selected. As a result, an assembly tree 2601 is displayed”). The same motivation of claim 3 applies here.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yano et al. (US 20110210868) in view of MURAKAMI (US 20250252646), and further in view of Morita et al. (US 20070132662) and Timoneda et al. (US 20180093613).
Regarding claim 5, Yano in view of MURAKAMI and Morita discloses The method according to claim 4.
On the other hand, Yano in view of MURAKAMI and Morita fails to explicitly disclose but Timoneda discloses wherein the extra-information comprises information which is stationary with respect to the vehicle, in particular distance information regarding the vehicle (Timoneda, “[0019] a display integrated in the rear-view mirror between the rear cover and the transflective element, the display being configurable for displaying an image of a first rear field of view of the rear part of the vehicle including ground level and captured by at least a first camera located at the vehicle, wherein the image to be displayed by the display is modified by processing means configured to overlay on said image at least a reference line which corresponds to a predetermined distance at ground level from the vehicle”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined Timoneda into the combination of Yano in view of MURAKAMI and Morita, to include all limitations of claim 5. That is, applying the overlay at a predetermined distance from the vehicle of Timoneda to the display system of Yano in view of MURAKAMI and Morita. The motivation/ suggestion would have been to provide design and construction of a driving assist system comprising cameras and display for use in vehicles to assist the driver in any steering operation, forward and reverse maneuvers (Timoneda, [0002]).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yano et al. (US 20110210868) in view of MURAKAMI (US 20250252646), and further in view of LEE et al. (US 20230304821).
Regarding claim 7, Yano in view of MURAKAMI discloses The method according to claim 1.
On the other hand, Yano in view of MURAKAMI fails to explicitly disclose but LEE discloses scaling the image data and/or the extra-information for display on the image display unit (LEE, fig.14, “[0309] Thus, as shown in FIG. 14B, a size the image of the building 1410 from the perspective of the vehicle gradually increases in response to the driving direction 1402 and the driving speed of the vehicle, and accordingly, the images of the plurality of pieces of AR digital signage 1421 and 1422 mapped to the building 1410 on a floor-by-floor basis are continuously corrected so that they can be increased in size and then mapped accordingly”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have combined LEE into the combination of MURAKAMI and Yano, to include all limitations of claim 7. That is, applying the size change principle of LEE to the display system of Yano and MURAKAMI. The motivation/ suggestion would have been to provide a digital signage platform providing apparatus capable of providing, to a driving image of a vehicle, augmented reality (AR) digital signage close to a real signboard (LEE, [0009]).
Response to Arguments
The objections to claims 6, 9 is/are withdrawn in view of amendments. The rejections under 35 USC § 112 to claims 1-9 are withdrawn.
Applicant’s arguments with respect to claim(s) 1-9 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GRACE Q LI whose telephone number is (571)270-0497. The examiner can normally be reached Monday - Friday, 8:00 am-5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, DEVONA FAULK can be reached at 571-272-7515. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GRACE Q LI/Primary Examiner, Art Unit 2618 5/27/2026
Prosecution Insights