DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-19 are pending and prosecuted.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-4, 6, 7, and 12-16 are rejected under 35 U.S.C. 103 as being unpatentable over Ha et al., US Patent Publication 2016/0225012, in further view of Squire et al., US Statutory Invention Registration, US H2282H.
Regarding Claim 1, Ha discloses a video method (Abstract; a method) comprising:
displaying one or more views from one or more video in one or more windows on a display of an operator client (Figure 3A-4E; [0060-0065] a client device 300 that displays a primary display area 304 and one or more secondary display areas 306);
detecting a gaze position of an operator viewing the display (Figure 3A-4E; [0060-0065]; [0070-0074]; a camera 308 that actively track the user’s eye movements and detect when the visual focus 302 of the user transitions from the primary display area 304 to the secondary display area);
determining a window, or a position within a window, from among the one or more windows corresponding to the gaze position (Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; detecting when the visual focus is one of the secondary display area); and
updating the display of the one or more views based on the gaze position (Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; emphasizing a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item, Figure 4B or 4C).
However, Ha doesn’t explicitly disclose the video method is a video surveillance method,
The one or more views are one or more camera views, and the one or more video is one or more video cameras.
Squire et al., US H2282H, discloses a apparatus and method for a display apparatus that displays a plurality of displays, each of the displays correspond to the visual output of security cameras (a video surveillance method; one or more camera views; one or more video cameras). Where the display apparatus further comprises of a eye tracker to observe changes in an operator’s line of sight, thereby inferring a change in the particular one of the plural devices the observer wishes to observe (Abstract; Figure 1 and 2; col. 3, ll. 32-67;).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the disclosure of Ha to further include the teachings of Squire in order to provide the video method is a video surveillance method, the one or more views are one or more camera views, and the one or more video is one or more video cameras. The motivation to combine these analogous arts is to provide a rapid and hands free switching among plural devices (Squire: Abstract;)
Regarding Claim 2, The combination of Ha and Squires teaches further comprising determining a camera view, or a position within a camera view, from among the one or more camera views corresponding to the gaze position based on the determined window, or position within a window (Ha: Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; Squires: Abstract; Figure 1 and 2; col. 3, ll. 32-67; emphasizing a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item, Figure 4B or 4C).
Regarding Claim 3, The combination of Ha and Squires teaches wherein the updating includes prioritising the display of the camera view, or a position within a camera view, corresponding to the gaze position (Ha: Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; Squires: Abstract; Figure 1 and 2; col. 3, ll. 32-67; emphasizing (prioritising) a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item, Figure 4B or 4C).
Regarding Claim 4, The combination of Ha and Squires teaches wherein the prioritising includes displaying the camera view corresponding to the gaze position at a larger size than the remaining camera views (Ha: Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; Squires: Abstract; Figure 1 and 2; col. 3, ll. 32-67; in response to detecting that the visual focus of the user meets a second threshold, the content management system may increase the size of the second content item to the size of the first display area, see Figure 4C).
Regarding Claim 6, The combination of Ha and Squires teaches wherein the prioritising includes displaying the camera view corresponding to the gaze position at a higher resolution than the remaining camera views (Ha: Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; Squires: Abstract; Figure 1 and 2; col. 3, ll. 32-67; in response to detecting that the visual focus of the user meets a second threshold, the content management system may increase the size of the second content item to the size of the first display area, see Figure 4C. The examiner considers the first display area to be a “higher resolution” than the other views due to the larger size).
Regarding Claim 7, The combination of Ha and Squires teaches wherein the prioritising includes highlighting an object at the position within the camera view that corresponds to the gaze position (Ha: Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; Squires: Abstract; Figure 1 and 2; col. 3, ll. 32-67; emphasizing (prioritizing; highlighting) a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item. As seen in Figure 4B, the content management system 100 can increase the size of the content item to at least partially cover one or more of the secondary display areas 406a-d and/or the primary display area 404 in response to detecting that the visual focus of the user transitions to the second item and remains for a length of time meeting a predetermined threshold).
Regarding Claim 12, The combination of Ha and Squires teaches a non-transitory computer-readable medium storing a computer program which, when executed by a programmable apparatus, causes the apparatus to perform the method of claim 1 (Ha: [0043]; a computer readable storage medium that comprises of one or more instructions that are executable by processors of one or more computing devices(e.g. client device), that cause the computing device to perform methods described herein)
Regarding Claim 13, Ha a video device (Abstract; a method, system and a display of a computing device) comprising:
an operator client including a display configured to display one or more views in one or more windows from one or more video (Figure 3A-4E; [0060-0065] a client device 300 that displays a primary display area 304 and one or more secondary display areas 306); and
an eye tracking device configured to detect a gaze position of an operator viewing the display, and provide the detected gaze position to the operator client (Figure 3A-4E; [0060-0065]; [0070-0074]; a camera 308 that actively track the user’s eye movements and detect when the visual focus 302 of the user transitions from the primary display area 304 to the secondary display area); wherein the operator client is configured to:
determine a window, or a position within a window, from among the one or more windows that corresponds to the gaze position(Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; detecting when the visual focus is one of the secondary display area); and
update the display of the one or more camera views on the display based on the gaze position(Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; emphasizing a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item, Figure 4B or 4C).
However, Ha doesn’t explicitly disclose the video method is a video surveillance method,
The one or more views are one or more camera views, and the one or more video is one or more video cameras.
Squire et al., US H2282H, discloses a apparatus and method for a display apparatus that displays a plurality of displays, each of the displays correspond to the visual output of security cameras (a video surveillance method; one or more camera views; one or more video cameras). Where the display apparatus further comprises of a eye tracker to observe changes in an operator’s line of sight, thereby inferring a change in the particular one of the plural devices the observer wishes to observe (Abstract; Figure 1 and 2; col. 3, ll. 32-67;).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the disclosure of Ha to further include the teachings of Squire in order to provide the video method is a video surveillance method, the one or more views are one or more camera views, and the one or more video is one or more video cameras. The motivation to combine these analogous arts is to provide a rapid and hands free switching among plural devices (Squire: Abstract;)
Regarding Claim 14, The combination of Ha and Squires teaches wherein the client device is further configured to determine a camera view, or a position within a camera view, from among the one or more camera views that corresponds to the gaze position based on the determined window, or position within a window (Ha: Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; Squires: Abstract; Figure 1 and 2; col. 3, ll. 32-67; emphasizing a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item, Figure 4B or 4C).
Regarding Claim 15, The combination of Ha and Squires teaches wherein the operator client is configured to update the display to prioritise the display of the camera view, or a position within a camera view, corresponding to the gaze position (Ha: Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; Squires: Abstract; Figure 1 and 2; col. 3, ll. 32-67; emphasizing (prioritising) a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item, Figure 4B or 4C).
Regarding Claim 16, , The combination of Ha and Squires teaches wherein the prioritising includes highlighting an object at the position within the camera view that corresponds to the gaze position (Ha: Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; Squires: Abstract; Figure 1 and 2; col. 3, ll. 32-67; emphasizing (prioritizing; highlighting) a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item. As seen in Figure 4B, the content management system 100 can increase the size of the content item to at least partially cover one or more of the secondary display areas 406a-d and/or the primary display area 404 in response to detecting that the visual focus of the user transitions to the second item and remains for a length of time meeting a predetermined threshold).
Claim(s) 5 are rejected under 35 U.S.C. 103 as being unpatentable over Ha et al., US Patent Publication 2016/0225012, in further view of Squire et al., US Statutory Invention Registration, US H2282H, in further view of Monroe, US Patent Publication 2002/0097322, and in further view of Duckworth et al., US Patent Publication 2012/0200658, henceforth known as Duckworth.
Regarding Claim 5, The combination of Ha and Squires doesn’t explicitly teach wherein the prioritising includes displaying the camera view corresponding to the gaze position at a position close to the operator.
Monroe, US Patent Publication 2002/0097322, teaches the use of monitoring stations that employ multiple display monitors, where a primary monitor displays map panes or control panels and the secondary monitors are used to display one or more video streams of the cameras (Figure 7 and 8; [0040-0042]; [0099-0104];).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention to modify the combinational disclosure of Ha and Squires to further include the teachings of Monroe such that there are a multiple display monitors. The motivation to combine these arts is to provide Monroe teaches that the system for displaying videos from different cameras can be employed by a single or multiple video screen monitor stations (Monroe: [0040];).
However, the combination of Ha, Squires, and Monroe doesn’t explicitly teach wherein the prioritising includes displaying the camera view corresponding to the gaze position at a position close to the operator.
Duckworth et al., US Patent Publication 2012/0200658, teaches a videoconference multipoint control unit that automatically generates display layouts for videoconference endpoints. As shown in Figures 4A and 4B, the MCU 106 transmits video streams to endpoint having four display devices, 402, 404, 406, and 408. Where display device 406 is depicts the full-screen display (only being used for) of the video stream of the current speakers, in Figure 4A. When the current speakers changes, as shown in Figure 4B, the display device 406 is updated to reflect the current speaker. While in the example is shows display device 406, any of the display devices can be selected (Abstract; Figure 4A and 4B; [0032-0035];).
Duckworth’s teachings of having a display layout with multiple display devices for videoconferencing in which any one of the display devices can be used and selected to display, on the entire display device, the current speaker would have been recognized by one skilled in the art, as applicable to the combinational disclose of Ha, Squires, and Monroe’s teachings of a monitoring system that utilizes a plurality of monitoring stations to view different cameras, and the results would have been predictable and resulted in having the display device, of the plurality of display devices, closest to the operator/user being used to display the first display area such that when the user’s visual focus meets a second threshold on a second content item, the second content item is displayed on the first display area, since both Duckworth and the combination of Ha, Squires, and Monroe teaches displaying one of a plurality of different cameras/videos in a display area that is larger than the others.
Therefore, the claimed subject matter of “wherein the prioritising includes displaying the camera view corresponding to the gaze position at a position close to the operator” would have been obvious to a person having ordinary skill in the art, before the effective filing date of the claimed invention.
Claim(s) 8, 10, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Ha et al., US Patent Publication 2016/0225012, in further view of Squire et al., US Statutory Invention Registration, US H2282H, in further view of Fu et al., US Patent 11678011, henceforth known as Fu.
Regarding Claim 8, The combination of Ha and Squires doesn’t explicitly teach wherein the updating includes prioritising the display of a camera view that does not correspond to the gaze position.
Fu et al., US Patent 11678011, teaches a video feed monitoring system that detects events within the video streams, and then moves the feed corresponding to the event to the large location 260a that displays the plurality of video streams of cameras (Figure 6 and 18; col. 24, ll. 4-46; col. 25, ll. 64-col. 27. 46; col. 64, ll. 49-col. 65. ll. 46;).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combinational disclosure of Ha and Squires to further include the teachings of Fu in order to provide wherein the updating includes prioritising the display of a camera view that does not correspond to the gaze position. The motivation to combine these arts is because Fu teaches determining the arrangement of the video streams based on detected events such that new events are moved to be more prominent, such as being displayed at the large location 260a as shown in Figure 6A (Figure 6A; col. 65, ll. 23-30;).
Regarding Claim 10, The combination of Ha, Squires, and Fu teaches wherein the prioritisation includes highlighting a camera view that does not correspond to the gaze position (Fu: Figure 6 and 18; col. 24, ll. 4-46; col. 25, ll. 64-col. 27. 46; col. 64, ll. 49-col. 65. ll. 46; the examiner considers moving the feed corresponding the detected event to be more prominent as “highlighting a camera view” that does not correspond to the “gaze position”).
Regarding Claim 17, The combination of Ha and Squires doesn’t explicitly teach wherein the operator client is configured to update the display to prioritise the display of a camera view that does not correspond to the gaze position.
Fu et al., US Patent Publication 11678011, teaches a video feed monitoring system that detects events within the video streams, and then moves the feed corresponding to the event to the large location 260a that displays the plurality of video streams of cameras (Figure 6 and 18; col. 24, ll. 4-46; col. 25, ll. 64-col. 27. 46; col. 64, ll. 49-col. 65. ll. 46;).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combinational disclosure of Ha and Squires to further include the teachings of Fu in order to provide wherein the operator client is configured to update the display to prioritise the display of a camera view that does not correspond to the gaze position. The motivation to combine these arts is because Fu teaches determining the arrangement of the video streams based on detected events such that new events are moved to be more prominent, such as being displayed at the large location 260a as shown in Figure 6A (Figure 6A; col. 65, ll. 23-30;).
Regarding Claim 18, Ha discloses a video system (Abstract; a method, system and a display of a computing device) comprising:
an operator client configured to receive, the plurality of video data streams and display the plurality of video data streams as a plurality of views in a plurality of windows on a display (Figure 3A-4E; [0060-0065] a client device 300 that displays a primary display area 304 and one or more secondary display areas 306); and
an eye tracking device configured to detect a gaze position of an operator viewing the display, and provide the detected gaze position to the operator client (Figure 3A-4E; [0060-0065]; [0070-0074]; a camera 308 that actively track the user’s eye movements and detect when the visual focus 302 of the user transitions from the primary display area 304 to the secondary display area); wherein the operator client is configured to:
determine a window, or a position within a window, from among the plurality of windows corresponding to the gaze position (Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; detecting when the visual focus is one of the secondary display area); and
update the display of the plurality of views on the display based on the gaze position (Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; emphasizing a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item, Figure 4B or 4C).
However, Ha doesn’t explicitly disclose the video method is a video surveillance method,
A plurality of video cameras,
a recording server configured to receive a plurality of video data streams from the plurality of video cameras; and
the one or more views are one or more camera views.
Squire et al., US H2282H, discloses a apparatus and method for a display apparatus that displays a plurality of displays, each of the displays correspond to the visual output of security cameras (a video surveillance method; a plurality of video cameras one or more camera views; one or more video cameras). Where the display apparatus further comprises of a eye tracker to observe changes in an operator’s line of sight, thereby inferring a change in the particular one of the plural devices the observer wishes to observe (Abstract; Figure 1 and 2; col. 3, ll. 32-67;).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the disclosure of Ha to further include the teachings of Squire in order to provide the video method is a video surveillance method, the one or more views are one or more camera views, and the one or more video is one or more video cameras. The motivation to combine these analogous arts is to provide a rapid and hands free switching among plural devices (Squire: Abstract;).
However, the combination of Ha and Squire doesn’t explicitly teach a recording server configured to receive a plurality of video data streams from the plurality of video cameras.
Fu et al., US Patent 11678011, teaches a video feed monitoring app, in which a remote device 104 is configured to communicate between smart devices 102a-102n, which can be cameras, and the user devices 104a-106n via the network. The remote device 104 is also used to provide processing and/or mass storage for the system, and is configured to store data, retrieve and transmit stored data, process data and/or communicate with other devices (Abstract; Figure 2; col. 5, ll. 29-col. 6, ll. 47;)
It would have bene obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combinational disclosure of Ha and Squire to further include the teachings of Fu in order to provide a recording server configured to receive a plurality of video data streams from the plurality of video cameras. The motivation to combine these arts is because Fu describes the use of a remote device 104 that is used to provide processing and/or mass storage for a system used for a video feed monitoring app (Abstract; col. 5, ll. 29-col. 6, ll. 47;).
Regarding Claim 19, The combination of Ha, Squire, and Fu teaches wherein the operator client is further configured to determine a camera view, or a position within a camera view, from among the one or more camera views that corresponds to the gaze position based on the determined window, or position within a window (Ha: Figure 3A-4E; [0060-0065]; [0070-0079]; [0095-0111]; Squires: Abstract; Figure 1 and 2; col. 3, ll. 32-67; emphasizing a second content item in one of the secondary display areas 406a-d when it detects focus of the user meets a predetermined threshold with respect to a content item, Figure 4B or 4C)..
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over Ha et al., US Patent Publication 2016/0225012, in further view of Squire et al., US Statutory Invention Registration, US H2282H, in further view of Fu et al., US Patent 11678011, henceforth known as Fu, and in further view of Cline et al., US Patent 9457914, henceforth known as Cline.
Regarding Claim 9, The combination of Ha, Squires, and Fu doesn’t explicitly teach wherein the prioritisation includes moving the display of a camera view that does not correspond to the gaze position to a position close to the gaze position.
Cline et al., US Patent 9457914, teaches a system and method for a eye tracking system that is configured to detect the current view of the pilot, and presenting the highest priority alert depending on where the pilot is current looking, so that it is within the current view of the pilot (Abstract; col. 2, ll. 26-45; col. 2, ll. 59-col. 3, ll. 16 ).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combinational disclosure of Ha, Squires, and Fu to further include the teachings of Cline such that a detected event caused the video feed to be presented into the current view of the pilot in order to provide wherein the prioritisation includes moving the display of a camera view that does not correspond to the gaze position to a position close to the gaze position. The motivation to combine these arts is because Cline teaches presenting alerts to the user so that it is within the current view of the user (Cline: Abstract; col. 2, ll. 26-45; col. 2, ll. 59-col. 3, ll. 16 ).
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Ha et al., US Patent Publication 2016/0225012, in further view of Squire et al., US Statutory Invention Registration, US H2282H, in further view of Monroe, US Patent Publication 2002/0097322, and in further view of Skulina et al., US Patent Publication 2019/0278365, henceforth known as Skulina.
Regarding Claim 11, The combination of Ha and Squires doesn’t explicitly teach wherein the method further comprises displaying a map view in a window on the display of the operator client,
wherein: the determining includes determining a position on the map view displayed in the window corresponding to the gaze position based on determined position within the window.
Monroe et al., US Patent Publication 2002/0097322, teaches a monitoring system in which a primary screen map window contains a map of the facility, where each map contains icons representing cameras or other sensor sites. Each camera/sensor icon represents the position of the camera within the facility. In addition, a camera icon are styled so as to indicate the direction the camera is pointed. When a mouse pointer dwells over a camera icon for a brief, predefined interval, a “bubble” appears identifying the camera. If the camera icon is double left clicked, then the camera’s video appears on the primary screen video window in a full screen view. The user can also drag and drop a camera icon into any unused pane in the primary screen video window to cause the selected video to appear in the selected ([0019-0021];).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combinational disclosure of Ha and Squires to further include the teachings of Monroe in order to provide wherein the method further comprises displaying a map view in a window on the display of the operator client. The motivation to combine these analogous arts is to provide a monitoring system that provides a map of the facility with icons representing the position of the cameras and that also indicate their direction (Monroe: [0019-0022];).
However, the combination of Ha, Squires, and Monroe doesn’t explicitly teach wherein: the determining includes determining a position on the map view displayed in the window corresponding to the gaze position based on determined position within the window.
Skulina et al., US Patent Publication 2019/0278365, teaches a computer apparatus in which a user may direct his gaze to a button in a graphical user interface on the display, where the button may be pressed in dependent on facial expression recognition instead of position a cursor over the button and clicking a mouse ([0015];).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the combinational disclosure of Ha, Squires, and Monroe to further include the teachings of Skulina such that the user can direct their gaze to the icons representing cameras on a map in order to select said camera for display in order to provide wherein: the determining includes determining a position on the map view displayed in the window corresponding to the gaze position based on determined position within the window. The motivation to combine these arts is because Skulina teaches that replacing position a cursor being positioned over a button and clicking a mouse with having a user direct their gaze to a button in a graphical user interface and detecting the users facial expression in order to press said button (Skulina: [0015];).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK F MARINELLI whose telephone number is (571)270-3383. The examiner can normally be reached Monday - Friday: 8:00AM - 5:00PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, George Eng can be reached at (571)-272-7495. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK F MARINELLI/Primary Examiner, Art Unit 2699