DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-24 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Lim et al (US 2020/0045261), Matsuzaka (US 2009/0245655), Parikh et al (US 2021/0132769) OR Chow et al (US Pat. 9,619,723).
Claims 1, 23 and 24, Lim teaches a method, a non-transitory medium and an electric device comprising:
receiving, via a camera that is in communication with a computer system, data representing a view of the camera; (see Figs. 3F and 3G for view of a single camera, i.e., camera 110b);
identifying, based on the data, a first subject as a primary subject in the view of the camera; (Fig. 3F and 3G, primary subject, i.e., participant 132, in the view of the camera 110b);
after identifying the first subject as the primary subject in the view of the camera and while the view of the camera consists of a first area of a physical environment, detecting that a focus direction of the first subject is in a second direction; (Fig. 3F shows, the participant 132 is on the left area of the view with a focus direction); and
in response to detecting that the focus direction of the first subject is in the second direction, shifting the view of the camera such that the view of the camera consists of a second area of the physical environment different from the first area of the physical environment. (Fig. 3G shows the participant in the right area of the view with a different focus direction. In another word, participant 132’s initial position in Fig. 3F is transition to a new physical position in Fig. 3G. Clearly the physical area on the left is different from the right).
Claims 1, 23 and 24, Matsuzaka teaches a method, a non-transitory medium and an electric device comprising:
receiving, via a camera that is in communication with a computer system, data representing a view of the camera; (a `front direction` means that a face looks directly at an imaging surface of an image generating apparatus, such as a digital still camera, [0047]);
identifying, based on the data, a first subject as a primary subject in the view of the camera; (a `front direction` means that a face looks directly at an imaging surface of an image generating apparatus, such as a digital still camera, [0047]);
after identifying the first subject as the primary subject in the view of the camera and while the view of the camera consists of a first area of a physical environment, detecting that a focus direction of the first subject is in a second direction; (a `right direction` means that a face turns to the right side of the imaging surface (the image of a face that turns to the left side when a viewer views the image), [0047]); and
in response to detecting that the focus direction of the first subject is in the second direction, shifting the view of the camera such that the view of the camera consists of a second area of the physical environment different from the first area of the physical environment. (a `left direction` means that a face turns to the left side of the imaging surface (the image of a face that turns to the right side when a viewer views the image), [0047]. Clearly a movement of right direction to the left direction indicates the different physical target or physical area).
Claims 1, 23 and 24, Parikh teaches a method, a non-transitory medium and an electric device comprising:
receiving, via a camera that is in communication with a computer system, data representing a view of the camera; (user 2702 is located within a field of view (FoV) of camera 2714 and thus, image sensor data generated by camera 2714 can include data representing user 2702, [0295]; In one or more embodiments, image sensor data generated by camera 2714 can be analyzed by a vision-based analyzer IC (e.g., 2540A, 2540B, 2540C) to determine whether a user is present in the camera's imaging field of view (FoV) as described with reference to FIG. 26, [0296]);
identifying, based on the data, a first subject as a primary subject in the view of the camera; (user 2702 is located within a field of view (FoV) of camera 2714 and thus, image sensor data generated by camera 2714 can include data representing user 2702, [0295]; In one or more embodiments, image sensor data generated by camera 2714 can be analyzed by a vision-based analyzer IC (e.g., 2540A, 2540B, 2540C) to determine whether a user is present in the camera's imaging field of view (FoV) as described with reference to FIG. 26, [0296]);
after identifying the first subject as the primary subject in the view of the camera and while the view of the camera consists of a first area of a physical environment, detecting that a focus direction of the first subject is in a second direction; (Via FIG. 27A, the vision-based analyzer IC may determine that user 2702 is present and engaged with respect to display panel 2712 based on determining that the user's face orientation 2704A is within a first-level area 2718, [0300]). Via FIG. 27B… the user's face orientation 2704B or 2704C is within a second-level area 2728B or 2728C, [0301]); and
in response to detecting that the focus direction of the first subject is in the second direction, shifting the view of the camera such that the view of the camera consists of a second area of the physical environment different from the first area of the physical environment. (Via FIG. 27A, the vision-based analyzer IC may determine that user 2702 is present and engaged with respect to display panel 2712 based on determining that the user's face orientation 2704A is within a first-level area 2718, [0300]). Via FIG. 27B… the user's face orientation 2704B or 2704C is within a second-level area 2728B or 2728C, [0301]); Clearly, first-level area 2718 is different from the other two second-level areas 2728B or 2728A).
Claims 1, 23 and 24, Chow teaches a method, a non-transitory medium and an electric device comprising:
receiving, via a camera that is in communication with a computer system, data representing a view of the camera; (Subject within the camera’s view plane, col. 5, lines 62-64);
identifying, based on the data, a first subject as a primary subject in the view of the camera; (Subject within the camera’s view plane, col. 5, lines 62-64);
after identifying the first subject as the primary subject in the view of the camera and while the view of the camera consists of a first area of a physical environment, detecting that a focus direction of the first subject is in a second direction; (prompting the subject to make a randomly generated sequence of movements including, but not limited to, moving, within the camera's view plane, her face (or head) towards a particular direction (e.g. laterally left or right, vertically up or down, diagonally, etc.), and turning (or rotating) her head towards a particular direction; and capturing the movements made by the subject. Col. 5, line 60 to col. 6, line 1); and
in response to detecting that the focus direction of the first subject is in the second direction, shifting the view of the camera such that the view of the camera consists of a second area of the physical environment different from the first area of the physical environment. (prompting the subject to make a randomly generated sequence of movements including, but not limited to, moving, within the camera's view plane, her face (or head) towards a particular direction (e.g. laterally left or right, vertically up or down, diagonally, etc.), and turning (or rotating) her head towards a particular direction; and capturing the movements made by the subject. Col. 5, line 60 to col. 6, line 1. Here clearly, there is a movement of the subject face from one position to another within the camera’s view, yet different target areas).
Claims 2-3 and 19 wherein shifting the view of the camera (Per the current Specs, [0023]: In some examples, a sensor includes a combination of multiple sensors. In some examples, sensor data is captured by fusing data from one sensor with data from one or more other sensors. Although a single computer system is shown in FIG. 1, compute system 100 can also be implemented as two or more compute systems operating together.
Similarly, Lim shows multiple cameras in an array and In response to the participant 155 shifting to the gaze direction 902b to look at the participant 134, the video conferencing system 102 may select the RGB camera 180b as the foreground camera, [0090]).
Claim 4. The method of claim 1, wherein the first subject is identified as the primary subject based on a proximity of the first subject to a center of the view of the camera. (Lim: determine when a subject has come into proximity to the multimedia communication device 100, estimate a distance between the multimedia communication device 100 and a subject, estimate a physical position of a subject relative to one or more of the RGB cameras 110, and/or identify discontinuities in a depth image and related depth image data used to aid image segmentation for a foreground subject in an image captured by one of the RGB cameras 110, [0032]).
Claim 5. The method of claim 1, wherein the first subject is identified as the primary subject based on a proximity of the first subject to the camera. (See claim 4).
Claim 6. The method of claim 1, wherein the first subject is identified as the primary subject based on an analysis of one or more previously captured images. (Chow: wherein the two or more images can be extracted from the captured images and/or video frames of the previously performed process step of continuous facial movement tracking test, col. 3, lines 56-59. Parikh: if an authenticated user's face is captured by a camera is stored in the security module 361).
Claim 7. The method of claim 1, wherein the first subject is identified as the primary subject based on a length of time that the first subject has been identified in the view of the camera. (Parikh: a topic detection module 1736 can determine a person, place, or other topic discussed in a time period (e.g., in the past 30 seconds, past 1 minute, past 5 minutes) prior to a query, and answer the query based on the topic, [0206]).
Claim 8. The method of claim 1, further comprising: while the first subject is identified as the primary subject, identifying that the first subject has not been in the view of the camera for a threshold amount of time; in response to identifying that the first subject has not been in the view of the camera for the threshold amount of time, identifying a second subject as the primary subject, wherein the second subject is different from the first subject; detecting, based on second data received via the camera, a focus direction of the second subject; and in response to detecting that the focus direction of the second subject is in a fourth direction, shifting the view of the camera. (Lim: the participant is currently looking at the participant 132 along the gaze direction 902a, and as a result, the current foreground camera for the participant 155 is the RGB camera 180d. In response to the participant 155 shifting to the gaze direction 902b to look at the participant 134, the video conferencing system 102 may select the RGB camera 180b as the foreground camera, [0090]).
Claim 9. The method of claim 1, wherein shifting the view of the camera includes moving the view of the camera in the second direction. (See the independent claims).
Claim 10. The method of claim 1, wherein shifting the view of the camera includes moving the view of the camera until the first subject is located at a predefined location within the view of the camera. (Lim: the threshold distance 302 and/or a shape of, and physical positions for, the foreground space 303 may be defined and/or adjusted by a user; for example, during a setup process, [0046]; the video conferencing system 102 is configured to determine a gaze direction for the participant 155 and select the RGB camera 180 most directly aligned with the gaze direction of the participant 155, [0090]).
Claim 11. The method of claim 1, wherein shifting the view of the camera includes moving the view of the camera until a subject in the second direction is in the view of the camera. (Lim; See Figs. 3F and 3G, top images. Or Figs. 5E and 5F, top images).
Claim 12. The method of claim 1, wherein shifting the view of the camera includes moving the view of the camera a predetermined amount in the focus direction. (Lim; See Figs. 3F and 3G, top images. Or Figs. 5E and 5F, top images).
Claim 13. The method of claim 1, wherein the computer system includes the camera. (See the independent claims).
Claim 14, The method of claim 1, wherein a third electronic device, different from the computer system, includes the camera. (See Lim, (where RGB cameras 180a, 180b, 180c, and 180d respectively correspond to the RGB cameras 110a, 100b, 100c, and 110d), [0076]).
Claim 15. The method of claim 1, wherein the view of the camera is shifted in response to detecting that the focus direction of the first subject changed from a first direction to the second direction. (See the independent claims).
Claim 16. The method of claim 1, further comprising: receiving, from another computer system different from the computer system, a respective pose corresponding to the other computer system; and in response to receiving the respective pose corresponding to the other computer system: in accordance with a determination that the respective pose is a first pose, shifting the (Chow: see at least Figs. 3 and 4 for different poses).
Claim 17. The method of claim 16, wherein the respective pose of the other computer system is received as part of a telecommunication call between the computer system and the other computer system. (Chow: see at least Figs. 3 and 4 for different poses).Lim: In the example illustrated in FIG. 1, the first multimedia communication device 100 may be referred to as a “local” device, and the second multimedia communication device 160 may be referred to as a “remote” device.”, [0027]).
Claim 18. The method of claim 16, further comprising: after shifting the first subject; and after detecting the movement of the first subject: in accordance with a determination that the computer system is in a first mode, shifting, based on the movement of the first subject, the (Parikh: In a first type of device-level low-power state, such as ones characterized as an “idle” or “standby” low-power state, the device can quickly transition from the low-power state to an active state in response to user input, hardware or network events. In a second type of device-level low-power state, such as one characterized as a “sleep” state, the device consumes less power than in the first type of low-power state and volatile memory is kept refreshed to maintain the device state. In a third type of device-level low-power state, such as one characterized as a “hibernate” low-power state, the device consumes less power than in the second type of low-power state, [0083]. Full display mode vs. partial display mode, [0213]).
Claim 20. The method of claim 1, wherein the computer system includes a movement component, and wherein shifting the (Parikh: [0293] In at least one embodiment, each camera is associated with an imaging field of view (FoV), and may include (or be operably coupled to) a suitable image sensor for detecting movement and/or light that may indicate the presence of a user in the camera's imaging FoV).
Claim 21. The method of claim 1, wherein the first subject is identified as the primary subject based on which subject is detected to be speaking. (Lim: [0094] At the time shown in FIG. 9, the participant 924 is currently speaking, and accordingly may be referred to as the “active speaker” in the video conferencing session).
Claim 22. The method of claim 1, wherein the first subject is identified as the primary subject based on which subject is gazing at the camera. (Lim: See the Title).
Response to Arguments
Applicant’s arguments with respect to claim(s) 8/5/26 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUNG-HOANG J. NGUYEN whose telephone number is (571)270-1949. The examiner can normally be reached Reg. Sched. 6:00-3:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 571-272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUNG-HOANG J NGUYEN/Primary Examiner, Art Unit 2691
