DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 4, 7-9, 11 are rejected under 35 U.S.C. 103 as being unpatentable over Ketels (US PG Publication 2024/0073510) in view of Morita (US PG Publication 2018/0188943) and Frojdh (US PG Publication 2014/0081521).
Regarding Claim 1, Ketels (US PG Publication 2024/0073510) discloses a display (computing device 102 [0023]) system for a vehicle (vehicle 100 [0023]), the display system comprising:
a display (in-vehicle infotainment device [0023]) device disposed in a passenger compartment (in-vehicle [0023]) of the vehicle, the display device comprising a display screen (display views [0023]; computing device 102 includes or is otherwise coupled to a display 230 [0027]);
at least one sensor (touch screen, camera, or proximity sensor [0033]) for sensing a user’s hand gestures (receive gestural input from a user [0033]);
and a controller (computing device 102 [0027]) in communication with the display device (includes or is coupled to a display 230 [0027]), the at least one sensor (computing device 102 operating user interface manager 308 [0030], configured to receive gestural input [0033]), and a plurality of imagers (cameras 104a-f [0029] coupled to the computing device 102 [0228]) configured to capture image data in a plurality of different fields of view (having fields of view 106a-f [0029]), wherein the controller (computing device 102 implementing external view controller 306 [0030]) is operable to select a desired view of the image data for display on the display screen (external view controller 306 updates the active camera view selection [0034]) in response to a user’s hand gesture as sensed by the at least one sensor (based on the gestural input [0034]).
Ketels does not disclose but Morita (US PG Publication 2018/0188943) teaches wherein the at least one sensor comprises a first sensor (sensor 18 portions SU and SD, Figs. 3-4, [0048]-[0049]) for sensing a vertical component of a user’s hand gesture (moves along the direction of the virtual line D1, Figs. 3-4, [0048]-[0049]) and a second sensor (sensor 18 portions SL and SR, Figs. 3-4, [0048]-[0049]) for sensing a horizontal component of a user’s hand gesture (gesture of moving a hand in the horizontal direction [0066]; i.e., moves along the line between SL and SR, Figs. 3-4, [0048]-[0049]);
wherein each of the first and second sensors comprise an array of proximity sensors (two proximity sensors SU, SD, Figs. 3-4, [0048]-[0049]; two proximity sensors SL, SR, Figs. 3-4, [0048]-[0049]);
Ketels does not disclose but Frojdh (US PG Publication 2014/0081521) teaches the wherein the array of proximity sensors of the first sensor is disposed linearly along one of a right edge and a left edge of the display screen (display 423 having proximity sensors along its edges [0132], i.e., left and right edges) and the array of proximity sensors of the second sensor is disposed linearly along one of a top edge and a bottom edge of the display screen (display 423 having proximity sensors along its edges [0132], i.e., top and bottom edges).
One of ordinary skill in the art before the application was filed would have been motivated to implement the gesture sensor of Ketels using the proximity sensor 18 of Morita because Morita teaches that implementing the gesture sensor using proximity sensor 18 enables the computing device to consistently perform a variety of processes via gestures and in response to various situations, and thus provides improved operability [0110].
One of ordinary skill in the art before the application was filed would have been motivated to implement the gesture sensor of Ketels with sensors around the display, as taught by Frodjh, because it is a known arrangement that enables detection of complex gestures [0023], improving the system.
Regarding Claim 4, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 1, wherein the at least one sensor further includes an imager (gesture tracking sensor is camera [0033]).
Regarding Claim 7, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 1, wherein the controller is further operable to process the image data by scaling, combining, and/or cropping (the active camera view may be generated by stitching together image data from multiple cameras 104 or otherwise combining the image data [0036]) in response to a user’s hand gesture received by the at least one sensor (based on the gestural input [0034]).
Regarding Claim 8, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 1, wherein the controller is further configured to change the display mode in response to vehicle operating parameters (activate the external camera view interface in response to a vehicle event such as the vehicle 100 changing into reverse gear [0035]).
Regarding Claim 9, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 8, wherein the vehicle operating parameters include vehicle speed, vehicle gear, turn signal activation, and proximity to an external object (activate the external camera view interface in response to a vehicle event such as the vehicle 100 changing into reverse gear [0035]).
Regarding Claim 11, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 1, wherein the controller is further configured to select which of a plurality of available fields of view to display (selecting a camera view adjacent to the active camera in the ordering of camera views [0034]) in response to a user’s hand gesture received by the at least one sensor (based on the gestural input [0034]).
Claim(s) 2 is rejected under 35 U.S.C. 103 as being unpatentable over Ketels (US PG Publication 2024/0073510) in view of Morita (US PG Publication 2018/0188943), Frojdh (US PG Publication 2014/0081521), and Niu (US PG Publication 2023/0271555).
Regarding Claim 2, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 1.
Ketels does not disclose but Niu (US PG Publication 2023/0271555) teaches wherein the display device is disposed in a rearview display assembly (the rearview apparatus 10 may correspond to a display mirror that is capable of displaying image data, such as a rearview display mirror [0028]).
One of ordinary skill in the art before the application was filed would have been motivated to integrate the display of Ketels into the rearview mirror of Ketel, as in Niu, because drivers have a habit of looking there, making it a convenient location to place captured images, especially when viewing rearward images. One of ordinary skill in the art before the application was filed would also have been motivated to include a camera for watching kids in the backseat, as in Niu, because parents who are driving their kids would like to be reassured that their kids are safe and behaving properly in the back seat [0022].
Claim(s) 5, 6 are rejected under 35 U.S.C. 103 as being unpatentable over Ketels (US PG Publication 2024/0073510) in view of Morita (US PG Publication 2018/0188943), Frojdh (US PG Publication 2014/0081521), and Zhu (US PG 2024/0187725).
Regarding Claim 5, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 1.
Ketels does not disclose but Zhu (US PG 2024/0187725) teaches wherein the controller is operable to select a desired display mode in which to display image data on the display screen in response to a user’s hand gesture received by the at least one sensor (in response to gesture, switch from full-screen to picture in picture [0109]).
One of ordinary skill in the art before the application was filed would have been motivated to supplement the user interface of Ketels to incorporate a picture-in-picture mode because Zhu teaches that it enables the user of the camera system to view images from multiple cameras at once [0109], providing a better user experience.
Regarding Claim 6, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 5.
Ketels does not disclose but Zhu (US PG 2024/0187725) teaches wherein the controller is operable to select the desired display mode from a full display mode, a superimposed display mode, and a side-by-side display mode (in response to gesture, switch from full-screen to picture in picture [0109]).
One of ordinary skill in the art before the application was filed would have been motivated to supplement the user interface of Ketels to incorporate a picture-in-picture mode because Zhu teaches that it enables the user of the camera system to view images from multiple cameras at once [0109], providing a better user experience.
Claim(s) 10 is rejected under 35 U.S.C. 103 as being unpatentable over Ketels (US PG Publication 2024/0073510) in view of Morita (US PG Publication 2018/0188943), Frojdh (US PG Publication 2014/0081521), and Park (US PG 2024/0185639).
Regarding Claim 10, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 1.
Ketels does not disclose but Park (US PG 2024/0185639) teaches wherein the controller is further configured to control a sunroof of the vehicle in response to a user’s hand gesture received by the at least one sensor (the sunroof 520, and the side mirror 530 may receive a control command corresponding to a gesture [0044]).
One of ordinary skill in the art before the application was filed would have been motivated to enable the sunroof of Ketels to be gesture-operated as in Park because gesture-operated vehicle functions enable drivers to conveniently access features of the vehicle without removing their eyes from the road to search the control panel for buttons, improving driver safety.
Claim(s) 12 is rejected under 35 U.S.C. 103 as being unpatentable over Ketels (US PG Publication 2024/0073510) in view of Morita (US PG Publication 2018/0188943), Frojdh (US PG Publication 2014/0081521), Niu (US PG Publication 2023/0271555), and Hsiao (US PG Publication 2016/0274669).
Regarding Claim 12, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 1, … wherein the controller is operable to select images to display on the display screen from the images captured (selecting a camera view [0034]) by the rear [] imager (cameras 104c and f capture rear views, Fig. 1) in response to gestures by a [] passenger sensed (based on the gestural input [0034]).
Ketels does not disclose but Niu (US PG Publication 2023/0271555) teaches wherein the at least one sensor further includes a rear passenger imager for capturing images in a field of view encompassing rear passengers (rearview apparatus 10 may include a camera 16 or imager configured to capture image data in a field of view 18 within the passenger compartment 14 [0016]), and display on the display screen from the images captured by the rear passenger imager (the rearview apparatus 10 may correspond to a display mirror that is capable of displaying image data, such as a rearview display mirror [0028]).
Ketels does not disclose, but Hsiao (US PG Publication 2016/0274669) teaches wherein the at least one sensor further includes a rear passenger imager for capturing images in a field of view encompassing rear passengers (camera better positioned to capture gestures of the rear passenger, capture gestures of the rear passenger [0027]),
gestures by a rear passenger sensed using the images captured by the rear passenger imager (gestures of the rear passenger, capture gestures of the rear passenger [0027]).
One of ordinary skill in the art before the application was filed would also have been motivated to include a camera for watching kids in the backseat of Ketels, as in Niu, because parents who are driving their kids would like to be reassured that their kids are safe and behaving properly in the back seat [0022].
One of ordinary skill in the art before the application was filed would have been motivated to supplement Ketels with a rear passenger camera, as in Hsiao, and allow rear passengers to be the gesture command operator, as in Hsiao, because Hsiao teaches that distributing the power to control some functions to vehicle passengers reduces driver distraction and promotes vehicular safety [0003], improving the system.
Claim(s) 15, 17, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Ketels (US PG Publication 2024/0073510) in view of Hendricks (US PG Publication 2021/0129753), Morita (US PG Publication 2018/0188943), and Frojdh (US PG Publication 2014/0081521).
Regarding Claim 15, Ketels (US PG Publication 2024/0073510) discloses display device comprising a display screen (in-vehicle infotainment device displays views [0023]);
and a controller (computing device 102 [0027]) in communication with the display device (includes or is coupled to a display 230 [0027]), the at least one sensor (computing device 102 operating user interface manager 308 [0030], configured to receive gestural input [0033]), and a plurality of imagers (cameras 104a-f [0029] coupled to the computing device 102 [0228]) configured to capture image data in a plurality of different fields of view (having fields of view 106a-f [0029]), wherein the controller (computing device 102 implementing external view controller 306 [0030]) is operable to select a desired view of the image data for display on the display screen (external view controller 306 updates the active camera view selection [0034]) in response to a user’s hand gesture as sensed (based on the gestural input [0034]) ….
Ketels does not disclose but Hendricks (US PG Publication 2021/0129753) teaches a rearview display assembly (a rearview mirror display assembly 178 [0056]) disposed within a passenger compartment of the vehicle (a rearview mirror display assembly 178 [0056]);
a display device disposed in the rearview display assembly (a rearview mirror display assembly 178 [0056]).
Ketels does not disclose but Morita (US PG Publication 2018/0188943) teaches a first sensor (sensor 18 portions SU and SD, Figs. 3-4, [0048]-[0049]) comprising a first array of proximity (two proximity sensors SU, SD, Figs. 3-4, [0048]-[0049]) sensors for sensing a vertical component of a user’s hand gesture (moves along the direction of the virtual line D1, Figs. 3-4, [0048]-[0049]) …;
a second sensor (sensor 18 portions SL and SR, Figs. 3-4, [0048]-[0049]) comprising a first array of proximity sensors (two proximity sensors SU, SD, Figs. 3-4, [0048]-[0049]) for sensing a horizontal component of a user’s hand gesture (gesture of moving a hand in the horizontal direction [0066]; i.e., moves along the line between SL and SR, Figs. 3-4, [0048]-[0049]), …;
hand gesture as sensed by the first and second sensors (gesture of moving a hand in the horizontal direction by SL and SR [0066], Figs. 3-4, [0048]-[0049]; gesture of moving a hand in the vertical direction [0063] by sensors SU and SD, Figs. 3-4 [0048]-[0049]).
Ketels does not disclose but Frojdh (US PG Publication 2014/0081521) teaches the first array of sensors disposed along one of a right side and a left side of the rearview display assembly (display 423 having proximity sensors along its edges [0132], i.e., left and right edges);
the second array of sensors disposed along one of a top side and a bottom side of the rearview display assembly (display 423 having proximity sensors along its edges [0132], i.e., top and bottom edges).
One of ordinary skill in the art before the application was filed would have been motivated to implement the display of Ketels as a rear-view mirror display because the rear-view mirror is one of the known locations for displaying images to a driver, and drivers already have a habit of looking there, making the rear-view mirror an obvious choice for implementing an in-vehicle display.
One of ordinary skill in the art before the application was filed would have been motivated to implement the gesture sensor of Ketels using the proximity sensor 18 of Morita because Morita teaches that implementing the gesture sensor using proximity sensor 18 enables the computing device to consistently perform a variety of processes via gestures and in response to various situations, and thus provides improved operability [0110].
One of ordinary skill in the art before the application was filed would have been motivated to implement the gesture sensor of Ketels with sensors around the display, as taught by Frodjh, because it is a known arrangement that enables detection of complex gestures [0023], improving the system.
Regarding Claim 17, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 15, wherein the controller is further configured to change the display mode in response to vehicle operating parameters (activate the external camera view interface in response to a vehicle event such as the vehicle 100 changing into reverse gear [0035]).
Regarding Claim 18, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 15, wherein the controller is further configured to select which of a plurality of available fields of view to display (selecting a camera view adjacent to the active camera in the ordering of camera views [0034]) in response to a user’s hand gesture as sensed (based on the gestural input [0034]).
Ketels does not disclose but Morita (US PG Publication 2018/0188943) teaches gesture as sensed by the first and second sensors (gesture of moving a hand in the horizontal direction by SL and SR [0066], Figs. 3-4, [0048]-[0049]; gesture of moving a hand in the vertical direction [0063] by sensors SU and SD, Figs. 3-4 [0048]-[0049]).
One of ordinary skill in the art before the application was filed would have been motivated to implement the gesture sensor of Ketels using the proximity sensor 18 of Morita because Morita teaches that implementing the gesture sensor using proximity sensor 18 enables the computing device to consistently perform a variety of processes via gestures and in response to various situations, and thus provides improved operability [0110].
Claim(s) 16 is rejected under 35 U.S.C. 103 as being unpatentable over Ketels (US PG Publication 2024/0073510) in view of Hendricks (US PG Publication 2021/0129753), Morita (US PG Publication 2018/0188943), Frojdh (US PG Publication 2014/0081521), and Zhu (US PG 2024/0187725).
Regarding Claim 16, Ketels (US PG Publication 2024/0073510) discloses the display system according to claim 15.
Ketels does not disclose but Morita (US PG Publication 2018/0188943) teaches gesture as sensed by the first and second sensors (gesture of moving a hand in the horizontal direction by SL and SR [0066], Figs. 3-4, [0048]-[0049]; gesture of moving a hand in the vertical direction [0063] by sensors SU and SD, Figs. 3-4 [0048]-[0049]).
Ketels does not disclose but Zhu (US PG 2024/0187725) teaches wherein the controller is operable to select a desired display mode in which to display image data on the display screen in response to a user’s hand gesture as sensed (in response to gesture, switch from full-screen to picture in picture [0109]).
One of ordinary skill in the art before the application was filed would have been motivated to supplement the user interface of Ketels to incorporate a picture-in-picture mode because Zhu teaches that it enables the user of the camera system to view images from multiple cameras at once [0109], providing a better user experience.
One of ordinary skill in the art before the application was filed would have been motivated to implement the gesture sensor of Ketels using the proximity sensor 18 of Morita because Morita teaches that implementing the gesture sensor using proximity sensor 18 enables the computing device to consistently perform a variety of processes via gestures and in response to various situations, and thus provides improved operability [0110].
Claim(s) 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over He (CN 105564316) in view of Hsiao (US PG Publication 2016/0274669) and Niu (US PG Publication 2023/0271555).
Regarding Claim 19, He (CN 105564316) discloses a display system for a vehicle (vehicle blind spot monitoring system 100 [32], Fig. 1), the display system comprising:
a display device (display unit 120 [32]) disposed in a front portion of a passenger compartment of the vehicle (inferred: blind spot displays are inferred to be in a place where the driver can see them, i.e., in a front portion of the passenger compartment), the display device comprising a display screen visible to a driver of the vehicle (inferred: blind spot displays are inferred to be in a place where the driver can see them, i.e., in a front portion of the passenger compartment);
and a controller (processor 140 [32]) in communication with the display device (select a camera to be at display unit 120 [40]), … and a plurality of imagers (image acquisition unit 110 transmits the collected blind spot image back to the processor 140 [37]) configured to capture image data in a plurality of different fields of view (front left camera and a left rear camera [34]; right front camera and a right rear camera [34]; front camera, reverse camera [35]), wherein the controller is operable to select images to display on the display screen (select a camera to be at display unit 120 [40]) from the images captured (camera to be at display [40]) … in response to gestures (enter commands by gesture to manually select the camera [45]).
He does not disclose, but Hsiao (US PG Publication 2016/0274669) teaches at least one sensor (camera better positioned to capture gestures of the rear passenger [0027]) for sensing a user’s hand gestures (capture gestures of the rear passenger [0027]), the at least one sensor includes a rear passenger imager for capturing images in a field of view encompassing rear passengers (camera better positioned to capture gestures of the rear passenger [0027]);
a controller (processing device 202 [0021]) in communication with the at least one sensor (camera device 101 may be arranged in the cabin for shooting images of gestures from the gesture command operator and transmitting the gesture images to the processing device 202),
wherein the controller (vehicle operating system [0020]) is operable (any passenger can be the gesture command operator and operate any execution device [0021]) … from the images captured by the rear passenger imager (rear passenger seat 105 as the gesture command operator [0027]) in response to gestures by a rear passenger sensed (when the gesture command operator is switched to the passenger in the rear passenger seat 104 or the rear passenger seat 105 [0027]) using the images captured by the rear passenger imager (active camera device may be switched to another camera better positioned to capture gestures of the rear passenger [0027]).
He does not disclose but Niu (US PG Publication 2023/0271555) teaches select images to display on the display screen from the images captured by the rear passenger imager (display image data captured by one or more additional interior cameras within the vehicle, e.g., a passenger or child monitoring camera [0022]).
One of ordinary skill in the art before the application was filed would have been motivated to supplement He with a rear passenger camera, as in Hsiao, and allow rear passengers to be the gesture command operator, as in Hsiao, because Hsiao teaches that distributing the power to control some functions to vehicle passengers reduces driver distraction and promotes vehicular safety [0003], improving the system.
One of ordinary skill in the art before the application was filed would also have been motivated to include a camera for watching kids in the backseat of He, as in Niu, because parents who are driving their kids would like to be reassured that their kids are safe and behaving properly in the back seat [0022].
Regarding Claim 20, He (CN 105564316) discloses the display system of claim 19, wherein the controller is operable to select a desired view of the image data for display on the display screen (select a camera to be at display unit 120 [40]) in response to a driver’s hand gesture (driver may enter commands by gesture to manually select the camera [45]).
He does not disclose, but Hsiao (US PG Publication 2016/0274669) teaches wherein the at least one sensor further includes a sensor for sensing a driver’s hand gestures (when the driver is selected as the gesture command operator, the camera 301 can be directed at the driver for shooting [0025]).
One of ordinary skill in the art before the application was filed would have been motivated to supplement He with a driver gesture-sensor, as in Hsiao, because He teaches that adopting the driver’s choices for blind-spot monitoring [45] enables the driver to easily monitor blind spots and reduces the occurrence of traffic accidents [10], improving transportation safety.
Response to Arguments
Applicant’s remarks filed 7/28/2026 have been considered but do not apply to the combination of references applied in this office action, which are based on amended limitations.
Regarding claims 12 and 19, Applicant argues that there is no suggestion, among the combination of references, of using the rear-passenger gesture detecting camera to control the display of the rear-passenger images on the display visible to the driver. This is unpersuasive because the rejection is based on a combination of references. Ketels (primary reference for Claim 12) or He (primary reference for claim 19) use driver gestures to switch the feed among multiple vehicle cameras. Niu adds the feature of an additional camera watching the backseat, the stream of which is displayable on the driver’s display. The combination of Ketels and Niu engenders the gestures of the driver to select among camera feeds including the backseat camera. Hsiao, relied upon in this office action, teaches a camera better positioned to capture gestures from rear-passengers, and gives rear-passengers the power to be gesture operators. The combination of Ketels or He, and Hsiao, engenders the rear-passenger gesture operator to select among camera views. The combination of three references suggests that the rear passengers’ gestures can flip through camera views, including a view of the rear passengers.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US 20220169275 – vehicle having gesture detection capability
US 20220185306 – controlling a vehicle using gestures
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHADAN E HAGHANI whose telephone number is (571)270-5631. The examiner can normally be reached M-F 9AM - 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jay Patel can be reached at 571-272-2988. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHADAN E HAGHANI/ Examiner, Art Unit 2485
Prosecution Insights