DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission has been entered.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 7-11 and 15-16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Olleon (US 2013/00308411) in view of Kwatra (US 2022/0108624).
Regarding claim 1, Olleon discloses a vehicle (fig. 1) comprising:
a user monitoring device (40 in fig. 3 and para. 19; including internal camera 14 and gesture sensor 20) for monitoring a user viewing an environment (para. 47 and Abstract);
a display device (16, fig. 1);
one or more processors (24, fig. 2); and
a non-transitory computer-readable medium (28, fig. 2) storing instructions that, when executed by the one or more processors (para. 19), cause the one or more processors to:
receive user data automatically from the user monitoring device without user intervention (para. 19, 21 and fig. 3; wherein the correlation processor 42 actively/automatically monitors the user, further wherein the correlation processor operates continuously without requiring the user to speak, wherein user intervention is interpreted as the user speaking) that corresponds to one or more visual characteristics of the user (para. 21) including directions, facial expression, movement and physiological characteristics (para. 15, 29, 47);
determine, based on the user data, that the user did not comprehend a sign in the environment (para. 21); and
display text of the sign on the display device (para. 21 and fig. 8).
Olleon fails to disclose to determine comprehension of a user only on the visual characteristics of the user.
Kwatra discloses to determine, based on only the visual characteristics of the user data (see 205 in fig. 2 and para. 33; wherein user data includes e.g. only hand gestures, eye movement and posture data and e.g. excludes audio data), that the user did not comprehend a sign in the environment (para. 26-28, 34; wherein comprehension of what the user is currently reading is determined through visually sensed user data).
When the invention was made (pre-AIA ) or before the effective filing date of the claimed invention (AIA ), it would have been obvious to one of ordinary skill in the art to include the teachings of Kwatra in the device of Olleon. The motivation for doing so would have been to determine user comprehension of what a user is currently reading using only visual characteristics of a user that are captured by a camera (Kwatra; see para. 26-28, 34, wherein the device is able to accurately determine word comprehension in environments in which audio data is not reliable or is unusable).
Regarding claim 2, Olleon discloses wherein determining that the user did not comprehend a sign in the environment is performed by inputting the user data into a trained model that is trained to output a comprehension state (para. 23, 27, 29).
Regarding claim 3, Olleon discloses wherein the trained model is a classifier operable to output a yes comprehension state and a no comprehension state (para. 27-28, 34).
Regarding claim 7, Olleon discloses wherein the display device is an infotainment display device (para. 15).
Regarding claim 8, Olleon discloses wherein the instructions cause the one or more processors to prioritize display of road signs over billboard signs (see fig. 8 and Abstract; wherein the processors prioritize what the user is pointing at, e.g. a road sign instead of a billboard) without user intervention (fig. 8 and para. 42; wherein e.g. the pointing hand gesture prioritizes which sign the user is confused about when there are multiple signs, further wherein the generic language of “what is that sign” would not prioritize between multiple signs).
Claims 9-11 and 15-16 are rejected for the same reasons stated for claims 1-3 and 7-8, respectively. See above rejections.
Claim(s) 5-6 and 13-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Olleon in view of Kwatra in further view of Viswanathan (US 2020/0189390), hereinafter referred to as “Vis”.
Regarding claim 5, Olleon fails to disclose a heads up display.
Vis discloses a wherein the display device is a heads-up display (para. 39).
When the invention was made (pre-AIA ) or before the effective filing date of the claimed invention (AIA ), it would have been obvious to one of ordinary skill in the art to include the teachings of Vis in the device of Olleon. The motivation for doing so would have been to provide a transparent display screen on the driver’s windshield (Vis; para. 39), allowing the driver’s eyes to remain on the road while reading the displayed information.
Regarding claim 6, Vis discloses wherein the text is displayed on or below the sign (see fig. 3-4 and para. 15).
Claims 13-14 are rejected for the same reasons stated for claims 5-6, respectively. See above rejections.
Response to Arguments
Applicant’s arguments with respect to claims have been considered but are moot in view of new grounds of rejection. See new citations above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROBIN J MISHLER whose telephone number is (571)270-7251. The examiner can normally be reached 8:00-5:00 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, NITIN PATEL can be reached at (571)272-7677. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROBIN J MISHLER/ Primary Examiner, Art Unit 2628