DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of Claims
This Office Action is in response to the Applicant’s amendments and remarks filed 01 July 2026. The Applicant has amended claims 1, 4, 6-7, 11, 14, 16 and 19 and canceled claims 5, 8, 15 and 20. Claims 1-4, 6-7, 9-14 and 16-19 are presently pending and are presented for examination.
Reply to Applicant’s Remarks
Applicant’s remarks filed 01 July 2026 have been fully considered and are addressed as follows:
Claim Rejections under 35 U.S.C. 112:
Applicant’s amendments to the claims filed 01 July 2026 have overcome the 35 U.S.C. 112(b) rejections previously set forth.
Claim Rejections under 35 U.S.C. 101:
Applicant’s amendments to the claims filed 01 July 2026 have overcome the 35 U.S.C. 101 rejections previously set forth.
Claims Rejections under 35 U.S.C. 102/103:
Applicant’s arguments, see Arguments/Remarks, filed 01 July 2026, with regard to the rejections of claims 1-20 under 35 U.S.C. 102/103 have been fully considered but they are not persuasive.
Regarding the Applicant’s arguments that “…Hayakawa fails to teach or suggest Applicant’s amended claim limitations…Strandberg is…silent with respect to augmented reality…Beaurepaire…nowhere is taught or suggested as…rendering component onto one or more surfaces, nor control components, nor controlling a vehicle based on an input entered via a control component”, the Examiner respectfully disagrees.
Hayakawa teaches …In the driving modes of levels 0 to 5, contents to be displayed by the HUD apparatus 100 are input and set in advance by the HUD controller… When determining that the mode signal indicates the fully automatic driving mode (YES in step S10), the HUD controller 102 sets the HUD 104 in a state where predetermined contents corresponding to the fully automatic driving mode can be displayed… (Hayakawa para 0133-0140; also see Fig. 5), i.e. Hayakawa discloses rendering a subset of components on a display based on a driving mode. Strandberg further teaches the displayed components including control components (Strandberg Fig. 4, Fig. 5A, Fig. 5B) and the display is a touchscreen display by which user can select the graphical touch control (Strandberg para 0049-0050). When Strandberg is combined with Hayakawa, different components could be displayed for different driving modes for driving safety. Beaurepaire further teaches the display could be an augmented reality (AR), and/or a virtual reality (VR) view inside a vehicle, such as on a window… or in a display… (Beaurepaire para 0080), i.e. when given the information of Beaurepaire, the display of Hayakawa could be implemented as an AR interface (e.g. a display) so that the content is displayed onto the surface of the display. Therefore, the prior art discloses the claim limitations as recited and the prior art and rejections have been maintained.
Claims 11 and 16 recite similar languages as claim 1 and are rejected for similar reasons above.
With respect to the dependent claims 2-4, 6-7, 9-10, 12-14 and 16-19, the Applicant provides no additional arguments other than their dependency from the independent claims 1 and 11. Because independent claims 1 and 11 are not allowable, dependent claims 2-4, 6-7, 9-10, 12-14 and 16-19 are not allowable.
Claim Objections
Claims 1, 11 and 16 are objected to because of the following informalities:
Claim 1, line 4 from the bottom, “…an AR interface…” should read --…an augmented reality (AR) interface…;
Claims 11 and 16 recite similar language as claim 1 and are objected for similar reasons above.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 6-7, 9-10, 11-13 and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Hayakawa (US20200339147) in view of Strandberg (US20200257364) and further in view of Beaurepaire (US20200329342).
As to claims 1, 11 and 16, Hayakawa teaches a computer-implemented method, a system and a computer program product comprising:
determining a set of conditions for a vehicle, wherein the vehicle is at least partially automated (Hayakawa para 0078: … The displayed contents mainly indicate various pieces of traveling information effective for a driving operation of the driver…; para 0083-0085: The sensor 5 detects a traveling state of the vehicle 1 and outputs traveling state information to the vehicle controller; para 0118: … the fifth display region 20e can be used as back monitors also in the partial automatic driving mode (level 2) where monitoring a driving environment is required; also see para 0111, Fig. 23);
determining a driving mode for the vehicle, wherein the driving mode represents a level of automation for the vehicle (Hayakawa para 0082: The vehicle controller 3 generates a mode signal indicating a driving mode of the vehicle…; also see para 0085);
analyzing the set of conditions and the driving mode (Hayakawa para 0133-0140: In the driving modes of levels 0 to 5, contents to be displayed by the HUD apparatus 100 are input and set in advance by the HUD controller for each of the driving models…; para 0110: the HUD controller 102 generates contents (see FIG. 5) indicating pieces of information based on various pieces of information obtained via the vehicle controller 3 (a vehicle speed, a time point, a driving mode, a video obtained by the camera 6, the sensor 5, the radar 7 or the like, other information, and the like). The HUD controller 102 drives the HUD 104 so as to execute a HUD display processing of displaying predetermined contents corresponding to a driving mode of the vehicle 1 in the display region 20…; also see para 0042: determines whether to change a driving mode…based on information detected by a sensor…contents indicating the reason for the change…are displayed together with the mode change notice display…);
determining, in response to the analyzing, a subset of components to add to a display, wherein each of the components is selected from a set of available components associated with the vehicle (Hayakawa para 0133-0140: In the driving modes of levels 0 to 5, contents to be displayed by the HUD apparatus 100 are input and set in advance by the HUD controller… When determining that the mode signal indicates the fully automatic driving mode (YES in step S10), the HUD controller 102 sets the HUD 104 in a state where predetermined contents corresponding to the fully automatic driving mode can be displayed…; also see Fig. 5),
rendering, the subset of components onto one or more surfaces within the vehicle (Hayakawa para 0133-140: When determining that the mode signal indicates the fully automatic driving mode (YES in step S10), the HUD controller 102 sets the HUD 104 in a state where predetermined contents corresponding to the fully automatic driving mode can be displayed; also see para 0014-0018; para 0028-0029).
Hayakawa further teaches microcomputer mainly including a CPU and a memory… vehicle control programs… (Hayakawa para 0109, para 0167).
Hayakawa does not teach analyzing, by a learning model… wherein the subset of components includes at least one control component; rendering, via an AR interface, the subset of components onto one or more surfaces within the vehicle; responsive to receiving an input from a driver via the at least one control component, controlling the vehicle based on the input.
Strandberg is directed to techniques for interacting with a vehicle touchscreen. Strandberg teaches …the display control component 904 can determine and/or receive information (e.g., from the machine learning component 1102) that defines the relative importance of available applications and/or functions to a particular user (e.g., driver) interfacing with the touchscreen 102 … the machine learning component 1102 can learn user preferences regarding relative importance of respective applications and/or functions in different contextual scenarios. These learned preferences can be used by the machine learning component 1102 and/or the display control component 904 to determine and/or infer what applications and/or functions to display… (Strandberg, para 0073, para 0081-0082, claim 12) …wherein the subset of components includes at least one control component, responsive to receiving an input from a driver via the at least one control component, controlling the vehicle based on the input (Strandberg Fig. 4, Fig. 5 and related text).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hayakawa so as to include analyzing, by a learning model…wherein the subset of components includes at least one control component, responsive to receiving an input from a driver via the at least one control component, controlling the vehicle based on the input in view of Strandberg et al. with a reasonable expectation of success. Those who have ordinary skill in the art would understand that learning model of Strandberg can be used in Hayakawa to determine the components to be displayed and the displayed components can be used for controlling, as required by the claim. One of ordinary skill would have been motivated to combine Hayakawa and Strandberg because this is merely combining prior art elements according to known methods to yield predictable results (KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007)).
Beaurepaire is directed to presenting contextual content. Beaurepaire teaches the system 100 presents the contextual content associated with a destination using any interface available on the vehicle 101 including but not limited to… augmented reality (AR), and/or a virtual reality (VR) view inside a vehicle, such as on a window… or in a display… (Beaurepaire para 0080).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hayakawa so as to include rendering, via an AR interface, the subset of components onto one or more surfaces within the vehicle in view of Beaurepaire et al. with a reasonable expectation of success. One of ordinary skill would have been motivated to combine Hayakawa and Beaurepaire because this is merely combining prior art elements according to known methods to yield predictable results (KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007)).
As to claims 2, 12 and 17, Hayakawa in view of Strandberg and Beaurepaire teaches the computer-implemented method of claim 1, the system of claim 11 and the computer program product of claim 16.
Hayakawa further teaches wherein the set of conditions includes external conditions outside the vehicle and internal conditions (Hayakawa para 0078: … The displayed contents mainly indicate various pieces of traveling information effective for a driving operation of the driver…; para 0083-0085: The sensor 5 detects a traveling state of the vehicle 1 and outputs traveling state information to the vehicle controller; para 0118: … the fifth display region 20e can be used as back monitors also in the partial automatic driving mode (level 2) where monitoring a driving environment is required; Fig. 23).
As to claims 3, 13 and 18, Hayakawa in view of Strandberg and Beaurepaire teaches the computer-implemented method of claim 2, the system of claim 12 and the computer program product of claim 17.
Hayakawa further teaches wherein the external conditions include weather, road conditions, traffic, road type, and location (Hayakawa para 0083-0086: …traveling state of the vehicle… outside weather condition…road shapes…GPS obtains current position information…).
As to claim 6, Hayakawa in view of Strandberg and Beaurepaire teaches the computer-implemented method of claim 1.
Beaurepaire further teaches wherein the AR interface is embedded into the vehicle (Beaurepaire para 0080).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hayakawa so as to include wherein the AR interface is embedded into the vehicle in view of Beaurepaire et al. with a reasonable expectation of success. One of ordinary skill would have been motivated to combine Hayakawa and Beaurepaire because this is merely combining prior art elements according to known methods to yield predictable results (KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007)).
As to claim 7, Hayakawa in view of Strandberg and Beaurepaire teaches the computer-implemented method of claim 1.
Beaurepaire further teaches wherein the AR interface is on an AR device communicatively connected to the vehicle (Beaurepaire para 0080).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hayakawa so as to include wherein the AR interface is on an AR device communicatively connected to the vehicle in view of Beaurepaire et al. with a reasonable expectation of success. One of ordinary skill would have been motivated to combine Hayakawa and Beaurepaire because this is merely combining prior art elements according to known methods to yield predictable results (KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007)).
As to claim 9, Hayakawa in view of Strandberg and Beaurepaire teaches the computer-implemented method of claim 1.
Hayakawa further teaches monitoring, in response to the rendering, the set of conditions; determining a change in at least one condition of the set of conditions; and updating, in response to the change, the subset of components, wherein the updating changes the subset of components (Hayakawa para 0110-0112: the HUD controller 102 generates contents (see FIG. 5) indicating pieces of information based on various pieces of information obtained via the vehicle controller 3 (a vehicle speed, a time point, a driving mode, a video obtained by the camera 6, the sensor 5, the radar 7…), also see para 0117-0118).
As to claim 10, Hayakawa in view of Strandberg and Beaurepaire teaches the computer-implemented method of claim 1.
Hayakawa further teaches wherein the vehicle includes a default subset of components and the determining the subset of components alters the default subset (Hayakawa para 0127: In the manual driving mode (level 0), for example, the content CO indicating the vehicle speed and the content C02 indicating the time point can be displayed. In the driving support mode (level 1), for example, the content C11 indicating a current driving mode of the vehicle 1 can be displayed in addition to the contents C0l and C02 that can be displayed in the manual driving mode).
Claims 4, 14 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Hayakawa in view of Strandberg and Beaurepaire as applied to claim 1 above, and further in view of Cheng (US20200193829).
As to claims 4, 14 and 19, Hayakawa in view of Strandberg and Beaurepaire teaches the computer-implemented method of claim 1, the system of claim 11 and the computer program product of claim 16.
Hayakawa further teaches identifying the driver of the vehicle (Hayakawa para 0083, para 0078, Fig. 23).
Hayakawa modified by Strandberg and Beaurepaire does not teach obtaining, in response to the identifying the driver, a profile for the driver including driver preferences and a driving history for the driver.
However, in the same field of endeavor, Cheng teaches…identifying the vehicle driver and the habits or driving history of the individual driver user or driver users of a vehicle; or, in the event of an autonomously driven vehicle, the habits of a passenger or passengers or the vehicle driving history… (Cheng para 0066).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Hayakawa so as to include obtaining, in response to the identifying the driver, a profile for the driver including driver preferences and a driving history for the driver in view of Cheng et al. with a reasonable expectation of success. One of ordinary skill would have been motivated to combine Hayakawa and Cheng because this is merely combining prior art elements according to known methods to yield predictable results (KSR International Co. v. Teleflex Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007)).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Examiner’s Notes
Examiner has cited particular columns/paragraph and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the examiner.
In the case of amending the claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention. This will assist in expediting compact prosecution. MPEP 714.02 recites: “Applicant should also specifically point out the support for any amendments made to the disclosure. See MPEP §2163.06. An amendment which does not comply with the provisions of 37 CFR 1.121(b), (c), (d), and (h) may be held not fully responsive. See MPEP § 714.” Amendments not pointing to specific support in the disclosure may be deemed as not complying with provisions of 37 C.F.R. 1.131(b), (c), (d), and (h) and therefore held not fully responsive. Generic statements such as "Applicants believe no new matter has been introduced" may be deemed insufficient.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HONGYE LIANG whose telephone number is (571)272-5410. The examiner can normally be reached on Monday-Friday 9:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rachid Bendidi can be reached on 571-272-4896. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HONGYE LIANG/Primary Examiner, Art Unit 3664
Prosecution Insights