DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-23 are rejected under 35 U.S.C. 102(a)(1) as anticipated by or, in the alternative, under 35 U.S.C. 103 as obvious over Lum (US 2023/0029764) and further in view of Avrahami et al (US 9,491,374).
Claims 1, 23 and 24, Lum teaches a method, a non-transitory medium and an electric device comprising::
receiving, via a camera that is in communication with a computer system, data representing a view of the camera; (Fig. 14, step 1402);
identifying, based on the data, a first subject as a primary subject in the view of the camera; (Fig. 14 step 1404);
after identifying the first subject as the primary subject in the view of the camera, detecting that a focus direction of the first subject is in a second direction; (Fig. 14, Step 1412 – and No, then go to step 1410); and
in response to detecting that the focus direction of the first subject is in the second direction, shifting the view of the camera. (While Fig 14 is “an example of another method for automatically switching a video feed from one camera to a video feed from another camera for display in a video conference” as Lum obtains the first video feed from the first camera; obtain the second video feed from the second camera; and display the first video feed from the first camera in a first area of a display based on a detection of a face in the first video feed and a gaze duration. And based on an absence of the face in the first video feed, display the face in the second video feed from the second camera.
Lum does not use the term “focus direction”, Lum does teach face detection, [0079-0084], gaze detection, [0118], gaze tracking, [0119] as well as gaze duration, [0025]. Here via Figs. 13 and 14, Lum clearly suggests, alternatively by obviousness, these activities cause/trigger a switch from one camera to another camera and thus the active spotlight is elevated.
To support this obviousness, Avrahami teaches “determining to switch the videoconference from the first camera to the second camera when the second camera has a view of the focus area with less obstruction and a higher degree of view than the first camera, col. 2, lines 61-64. Also see Avrahami’s claim 1. See Figs. 1 and 2.
Therefore it would have been obvious to the ordinary artisan before the effective filing date to explicitly describe the switching of the cameras when face direction presents the focus area with less obstruction and a higher degree of view than the first camera.
Claims 2-3 and 19 wherein shifting the view of the computer system includes moving a direction that the camera is facing; wherein moving the direction that the camera is facing includes sending, to a second computer system that is different from the computer system, a request to move a physical component of the second computer system. (Per the current Specs, [0023]: In some examples, a sensor includes a combination of multiple sensors. In some examples, sensor data is captured by fusing data from one sensor with data from one or more other sensors. Although a single compute system is shown in FIG. 1, compute system 100 can also be implemented as two or more compute systems operating together.
Similarly, Lum teaches, via fig. 8, multiple cameras, i.e., 802A, 802B and 802C, wherein each camera direct focusing specific target, [0079-0083]. However, second camera, like other cameras, can switch the focus directed to the host 804 OR the second inject 808, [0080]).
Claim 4. The method of claim 1, wherein the first subject is identified as the primary subject based on a proximity of the first subject to a center of the view of the camera. (Lum’s Figs. 13 and 14 for switching a video feed from one camera to a video feed from another camera for display in a video conference. See Fig. 4, Active spotlight in on the current speaker at the center of the display. [0063]).
Claim 5. The method of claim 1, wherein the first subject is identified as the primary subject based on a proximity of the first subject to the camera. (Lum: The dynamic bounding box 506 may be used to track the movement of an object in a video feed. The dynamic bounding box 506 may be automatically resized and/or relocated within the frame 500 based on a motion of the respective object and proximity to the camera, [0068]).
Claim 6. The method of claim 1, wherein the first subject is identified as the primary subject based on an analysis of one or more previously captured images. (Lum: At 1306, the method 1300 includes detecting an object or activity in the first video feed, the second video feed, or both. The object may be a person, a face, or another object, such as a knife in a cooking show example. Detecting an object may include identifying an area of the first video feed that may contain an object. The area of the first video feed may be identified using an AI algorithm trained for object detection. In some examples, the area may be identified based on a grouping of pixels, for example a grouping of differently colored pixels. A bounding box may be drawn around the identified area. The identified area may be classified as a particular object based on a probabilistic match when compared to stored objects in a database of objects, [0117]).
Claim 7. The method of claim 1, wherein the first subject is identified as the primary subject based on a length of time that the first subject has been identified in the view of the camera. (Lum: a participant video feed may be displayed in the automatic spotlight queue 404 if the associated relevance score meets a threshold, [0064-0066] or gaze duration, [0089-0091].
Claim 8. The method of claim 1, further comprising: while the first subject is identified as the primary subject, identifying that the first subject has not been in the view of the camera for a threshold amount of time; in response to identifying that the first subject has not been in the view of the camera for the threshold amount of time, identifying a second subject as the primary subject, wherein the second subject is different from the first subject; detecting, based on second data received via the camera, a focus direction of the second subject; and in response to detecting that the focus direction of the second subject is in a fourth direction, shifting the view of the camera. (Lum: a participant video feed may be displayed in the automatic spotlight queue 404 if the associated relevance score meets a threshold, [0064-0066] or gaze duration, [0089-0091] or Lum: video feed is elevated to an automatic spotlight when it is determined/scored at a certain threshold, [0063-0065] and Figs. 4A-4C. Specifically, FIG. 4A is a diagram of an example of a user interface output 400 to a display showing a video conference. The user interface output 400 may be displayed on a component of a client or a device, such as client 102 or phones 104 shown in FIG. 1, or clients 308, 310 and phone 312 shown in FIG. 3. In this example, the display 400 includes a host video feed 402 and an automatic spotlight queue 404. The automatic spotlight queue 404 is an area of the display that contains participant video feeds that have the potential to be elevated to an active spotlight status, [0063]. Here examiner reads that the spotlight is elevated or shifted if any of the video feeds 404A, 404B, 404C, 404D, and 404E, if any of these feeds meets the threshold).
Claim 9. The method of claim 1, wherein shifting the view of the camera includes moving the view of the camera in the second direction. (See the independent claims).
Claim 10. The method of claim 1, wherein shifting the view of the camera includes moving the view of the camera until the first subject is located at a predefined location within the view of the camera. (Lum’s Figs. 13 and 14 for switching a video feed from one camera to a video feed from another camera for display in a video conference. See Fig. 4, Active spotlight in on the current speaker at the center of the display. [0063]).
Claim 11. The method of claim 1, wherein shifting the view of the camera includes moving the view of the camera until a subject in the second direction is in the view of the camera. (Lum’s Figs. 13 and 14 for switching a video feed from one camera to a video feed from another camera for display in a video conference. See Fig. 4, Active spotlight in on the current speaker at the center of the display. [0063]).
Claim 12. The method of claim 1, wherein shifting the view of the camera includes moving the view of the camera a predetermined amount in the focus direction. (Lum: video feed is elevated to an automatic spotlight when it is determined/scored at a certain threshold, [0063-0065] and Figs. 4A-4C).
Claim 13. The method of claim 1, wherein the computer system includes the camera. (See the independent claims).
Claim 14, The method of claim 1, wherein a third electronic device, different from the computer system, includes the camera. (See Lum, Fig. 8).
Claim 15. The method of claim 1, wherein the view of the camera is shifted in response to detecting that the focus direction of the first subject changed from a first direction to the second direction. (See the independent claims or Lum’s Fig. 14).
Claim 16. The method of claim 1, further comprising: receiving, from another computer system different from the computer system, a respective pose corresponding to the other computer system; and in response to receiving the respective pose corresponding to the other computer system: in accordance with a determination that the respective pose is a first pose, shifting the field of view of the camera to a first field of view; and in accordance with a determination that the respective pose is a second pose different from the first pose, shifting the field of view of the camera to a second field of view different from the first field of view. (Lum: The video feeds from the first camera and the second camera may be obtained by the video conference system via a client, such as client 102 shown in FIG. 1 or clients 308 and 310 shown in FIG. 3. The first video feed from the first camera may have a first FOV and the second video from the second camera may have a second FOV. In some examples, the first FOV may partially overlap with the second FOV. In other examples, the first FOV and the second FOV may be non-overlapping FOVs, [0125]. Avrahami: (e.g., face detection and pose estimation) and …other areas within the field of view of the user's camera. Col, 4, line 21-33).
Claim 17. The method of claim 16, wherein the respective pose of the other computer system is received as part of a telecommunication call between the computer system and the other computer system. (See Lum’s Fig. 1 and Avrahami’s Fig. 4).
Claim 18. The method of claim 16, further comprising: after shifting the field of view to the first field of view, detecting, via the camera, movement of the first subject; and after detecting the movement of the first subject: in accordance with a determination that the computer system is in a first mode, shifting, based on the movement of the first subject, the field of view of the camera; and in accordance with a determination that the computer system is in a second mode different from the first mode, forgoing shifting, based on the movement of the first subject, the field of view of the camera. (Avrahami: (16) FIG. 2 illustrates an example view of a user engaged in a videoconference in a multi-display environment. Based on an input (e.g., head-pose, gaze-tracking, and/or activities associated with the user), it was determined that the user may be multitasking, by viewing information on the second display while the videoconference is displayed on the first display. Accordingly, the camera for the videoconference was switched from the camera of the first display to the camera of the second display. Accordingly, the user appears to be engaged in the videoconference. Col. 5, lines 23-32).
Claim 20. The method of claim 1, wherein the computer system includes a movement component, and wherein shifting the field of view of the camera includes controlling the movement component. (Lum: The video feeds from the first camera and the second camera may be obtained by the video conference system via a client, such as client 102 shown in FIG. 1 or clients 308 and 310 shown in FIG. 3. The first video feed from the first camera may have a first FOV and the second video from the second camera may have a second FOV. In some examples, the first FOV may partially overlap with the second FOV. In other examples, the first FOV and the second FOV may be non-overlapping FOVs. [0116]).
Claim 21. The method of claim 1, wherein the first subject is identified as the primary subject based on which subject is detected to be speaking. (Lum: a predetermined setting for a cooking class may be to display the video feed of preparing ingredients when significant motion is detected in the first video feed and display the second video feed when significant motion is not detected in the first video feed. In an example of a funeral, the predetermined setting may be to display the video feed of a speaker and occasionally switch the video feed to a video feed of the audience, [0118, 0122]).
Claim 22. The method of claim 1, wherein the first subject is identified as the primary subject based on which subject is gazing at the camera. (Lum: The gaze tracking can be used to determine which areas of the display participants are viewing. For example, the gaze tracking may be used to determine whether the participants are viewing the primary area of the display or the secondary area of the display, [0119]).
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUNG-HOANG J. NGUYEN whose telephone number is (571)270-1949. The examiner can normally be reached Reg. Sched. 6:00-3:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 571-272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUNG-HOANG J NGUYEN/Primary Examiner, Art Unit 2691
Prosecution Insights