DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
1. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
2. Claim(s) 1, 6 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 2022/0408015) in view of Hsin et al. (US 2025/0157186) in further view of Takahashi et al. (US 2017/0127017).
Regarding claim 1, Wang teaches a video conference system, which is connected to a remote user device via a network, the video conference system comprising: a master conference device, wherein the master conference device is configured to capture a master panoramic image that includes a speaker, recognize a master image of the speaker in the master panoramic image; and a slave conference device coupled to the master conference device, wherein the slave conference device is configured to capture a slave panoramic image that includes the speaker, recognize a slave image of the speaker in the slave panoramic image (see fig. 1A-6, 10, ¶ 0023, 0029-0036, 0064-0066. The videoconference room picks best camera view in a videoconference room when multiple cameras are installed. One camera is designated as the primary camera and includes a microphone array. The microphone array is used to figure out where speech is coming from. The system then combines that audio direction with the primary camera’s image to identify who is speaking. It estimates the speaker’s pose, such as face direction and keypoints. The other cameras also analyze the people they see and estimate each person’s pose. Those poses are compared to the speaker’s pose so the same person can be identified across cameras. After that, the system chooses the camera that gives the most frontal view of the speaker. In some versions, the system can also create a composite view made from the best views of a participants. The conferencing system has a primary camera and slave cameras.);
wherein the slave conference device transmits the slave image to the master conference device, the master conference device compares the master image with the slave image to generate a comparison result, and the master conference device transmits one of the master image and the slave image to the remote user device according to the comparison result (see fig. 7-10, ¶ 0023, 0029-0036, 0043-0045. 0064-0066. A speaker of a group of individuals or participants is identified using the sound source location and an image from the video stream. Pose information of the speaker is determined from the image from the video stream. Pose information of each individual in the group is determined by the plurality of cameras other than the primary camera which is compared to pose information of the speaker and best match and setting of the speaker is determined. A camera is selected from the plurality of cameras to provide a video stream the far end site based on the best frontal view of the speaker. The best image pose that is compared from multiple cameras are determined which is the best based on results and transmitting the video stream to the remote side.).
Wang discloses sending and receiving video streams from and to remote devices and framed face rectangle information that is compared with all cameras (see fig. 15A-15B, ¶ 0087-0089). Wang is vague on crop out the master image from the master panoramic image; and crop out the slave image from the slave panoramic image.
Hsin teaches crop out the master image from the master panoramic image; and crop out the slave image from the slave panoramic image (see fig. 1, ¶ 0012. Automatic people matching and selection system comprises a plurality of cameras (102) for capturing a plurality of images of members in a space and performing an artificial intelligence (AI) analysis on the images, a main computer, and a display. The AI analysis results such as member features and the corresponding cropped or full images are transmitted to corresponding buffers of the main computer to perform synchronization. Then the main computer performs a matching algorithm for generating a result matching the members captured by the plurality of cameras. The best appearances such as front faces of the members are then displayed on the display.)
The combination of Hsin to Wang provides for matching the best image that is cropped of members and using the best image to be displayed.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Wang to incorporate cropped images that are captured by cameras to find the best match between pictures of participants. The modification for finding the best match between the cropped images.
Takahashi discloses a panoramic image being generated based on the multiple cameras to generate the panoramic image for a conferencing system (see fig. 6, ¶ 0038).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Wang and Hsin to incorporate panoramic imaging. The modification provides for generating panoramic imaging with multiple cameras.
Regarding claim 6, Wang teaches a video conference method, comprising: capturing, by a master conference device, a master panoramic image that includes a speaker; recognizing, by the master conference device, a master image of the speaker in the master panoramic image;; capturing, by a slave conference device, a slave panoramic image that includes the speaker; recognizing, by the slave conference device, a slave image of the speaker in the slave panoramic image (see fig. 1A-6, 10, ¶ 0023, 0029-0036, 0064-0066. The videoconference room picks best camera view in a videoconference room when multiple cameras are installed. One camera is designated as the primary camera and includes a microphone array. The microphone array is used to figure out where speech is coming from. The system then combines that audio direction with the primary camera’s image to identify who is speaking. It estimates the speaker’s pose, such as face direction and keypoints. The other cameras also analyze the people they see and estimate each person’s pose. Those poses are compared to the speaker’s pose so the same person can be identified across cameras. After that, the system chooses the camera that gives the most frontal view of the speaker. In some versions, the system can also create a composite view made from the best views of a participants. The conferencing system has a primary camera and slave cameras.);
comparing, by the master conference device, the master image with the slave image to generate a comparison result; and transmitting, by the master conference device, one of the master image and the slave image to a remote user device according to the comparison result (see fig. 7-10, ¶ 0023, 0029-0036, 0043-0045. 0064-0066. A speaker of a group of individuals or participants is identified using the sound source location and an image from the video stream. Pose information of the speaker is determined from the image from the video stream. Pose information of each individual in the group is determined by the plurality of cameras other than the primary camera which is compared to pose information of the speaker and best match and setting of the speaker is determined. A camera is selected from the plurality of cameras to provide a video stream the far end site based on the best frontal view of the speaker. The best image pose that is compared from multiple cameras are determined which is the best based on results and transmitting the video stream to the remote side.).
Wang discloses sending and receiving video streams from and to remote devices and framed face rectangle information that is compared with all cameras (see fig. 15A-15B, ¶ 0087-0089). Wang is vague on cropping out, by the master conference device, the master image of the speaker from the master panoramic image; cropping out, by the slave conference device, the slave image of the speaker from the slave panoramic image.
Hsin teaches cropping out, by the master conference device, the master image of the speaker from the master panoramic image; cropping out, by the slave conference device, the slave image of the speaker from the slave panoramic image (see fig. 1, ¶ 0012. Automatic people matching and selection system comprises a plurality of cameras (102) for capturing a plurality of images of members in a space and performing an artificial intelligence (AI) analysis on the images, a main computer, and a display. The AI analysis results such as member features and the corresponding cropped or full images are transmitted to corresponding buffers of the main computer to perform synchronization. Then the main computer performs a matching algorithm for generating a result matching the members captured by the plurality of cameras. The best appearances such as front faces of the members are then displayed on the display.)
The combination of Hsin to Wang provides for matching the best image that is cropped of members and using the best image to be displayed.
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Wang to incorporate cropped images that are captured by cameras to find the best match between pictures of participants. The modification for finding the best match between the cropped images.
Takahashi discloses a panoramic image being generated based on the multiple cameras to generate the panoramic image for a conferencing system (see fig. 6, ¶ 0038).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Wang and Hsin to incorporate panoramic imaging. The modification provides for generating panoramic imaging with multiple cameras.
3. Claim(s) 2, 3, 7, 8 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 2022/0408015) in view of Hsin et al. (US 2025/0157186) in further view of Takahashi et al. (US 2017/0127017).
Regarding claim 2, Wang teaches the video conference system according to claim 1, wherein, when the comparison result indicates that the master image and the slave image have a same contour, the master conference device marks the master image and the slave image with a same identity label (see fig. 7-10, ¶ 0023, 0029-0036, 0043-0045. 0064-0066. A speaker of a group of individuals or participants is identified using the sound source location and an image from the video stream. Pose information of the speaker is determined from the image from the video stream. Pose information of each individual in the group is determined by the plurality of cameras other than the primary camera which is compared to pose information of the speaker and best match and setting of the speaker is determined. A camera is selected from the plurality of cameras to provide a video stream the far end site based on the best frontal view of the speaker. The best image pose that is compared from multiple cameras are determined which is the best based on results and transmitting the video stream to the remote side. Compared images pose boundaries developed, allow the setting of the best pose ID, the pose ID of the speaker, with its detailed pose information. The images area compared and provided with the ID which is a label for the best image.).
Regarding claim 3, Wang teaches the video conference system according to claim 2, wherein the master conference device determines whether or not facial features of the master image and facial features of the slave image meet frontal face criteria; wherein, when the facial features of the master image meet the frontal face criteria, the master conference device transmits the master image to the remote user device, and when the facial features of the slave image meet the frontal face criteria, the master conference device transmits the slave image to the remote user device (see fig. 1A-6, 10, ¶ 0023, 0029-0036, 0064-0066. The videoconference room picks best camera view in a videoconference room when multiple cameras are installed. One camera is designated as the primary camera and includes a microphone array. The microphone array is used to figure out where speech is coming from. The system then combines that audio direction with the primary camera’s image to identify who is speaking. It estimates the speaker’s pose, such as face direction and keypoints. The other cameras also analyze the people they see and estimate each person’s pose. Those poses are compared to the speaker’s pose so the same person can be identified across cameras. After that, the system chooses the camera that gives the most frontal view of the speaker. In some versions, the system can also create a composite view made from the best views of a participants. The conferencing system has a primary camera and slave cameras. The best image pose that is compared from multiple cameras are determined which is the best based on results and transmitting the video stream to the remote side.)
Regarding claim 7, Wang teaches the video conference method according to claim 6, further comprising: marking, by the master conference device, the master image and the slave image with a same identity label when the comparison result indicates that the master image and the slave image have a same contour (see fig. 7-10, ¶ 0023, 0029-0036, 0043-0045. 0064-0066. A speaker of a group of individuals or participants is identified using the sound source location and an image from the video stream. Pose information of the speaker is determined from the image from the video stream. Pose information of each individual in the group is determined by the plurality of cameras other than the primary camera which is compared to pose information of the speaker and best match and setting of the speaker is determined. A camera is selected from the plurality of cameras to provide a video stream the far end site based on the best frontal view of the speaker. The best image pose that is compared from multiple cameras are determined which is the best based on results and transmitting the video stream to the remote side. Compared images pose boundaries developed, allow the setting of the best pose ID, the pose ID of the speaker, with its detailed pose information. The images area compared and provided with the ID of the best image.).
Regarding claim 8, Wang teaches the video conference method according to claim 7, wherein the master conference device determines whether or not facial features of the master image and facial features of the slave image meet frontal face criteria; wherein, when the facial features of the master image meet the frontal face criteria, the master conference device transmits the master image to the remote user device, and when the facial features of the slave image meet the frontal face criteria, the master conference device transmits the slave image to the remote user device (see fig. 1A-6, 10, ¶ 0023, 0029-0036, 0064-0066. The videoconference room picks best camera view in a videoconference room when multiple cameras are installed. One camera is designated as the primary camera and includes a microphone array. The microphone array is used to figure out where speech is coming from. The system then combines that audio direction with the primary camera’s image to identify who is speaking. It estimates the speaker’s pose, such as face direction and keypoints. The other cameras also analyze the people they see and estimate each person’s pose. Those poses are compared to the speaker’s pose so the same person can be identified across cameras. After that, the system chooses the camera that gives the most frontal view of the speaker. In some versions, the system can also create a composite view made from the best views of a participants. The conferencing system has a primary camera and slave cameras. The best image pose that is compared from multiple cameras are determined which is the best based on results and transmitting the video stream to the remote side.)
4. Claim(s) 4, 9 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 2022/0408015) in view of Hsin et al. (US 2025/0157186) in further view of Takahashi et al. (US 2017/0127017) in further view of Duckworth et al. (US 2016/0173823).
Regarding claim 4, Wang teaches the video conference system according to claim 1, wherein at least one relative angle is formed between the speaker and at least one of the master conference device and the slave conference device; wherein at least one of the master conference device and the slave conference device receives voice data of a participant, and recognizes an orientation of the voice data of the participant, so as to generate master voice orientation data; wherein at least one of the master conference device and the slave conference device compares the at least one relative angle with the master voice orientation data; wherein, when the master voice orientation data matches the at least one relative angle, at least one of the master conference device and the slave conference device determines that the participant is the speaker (see fig. 1A-6, 10, ¶ 0023, 0029-0036, 0064-0066. The videoconference room picks best camera view in a videoconference room when multiple cameras are installed. One camera is designated as the primary camera and includes a microphone array. The microphone array is used to figure out where speech is coming from. The system then combines that audio direction with the primary camera’s image to identify who is speaking. It estimates the speaker’s pose, such as face direction and keypoints. The other cameras also analyze the people they see and estimate each person’s pose. Those poses are compared to the speaker’s pose so the same person can be identified across cameras. After that, the system chooses the camera that gives the most frontal view of the speaker. In some versions, the system can also create a composite view made from the best views of a participants. The conferencing system has a primary camera and slave cameras. The best image pose that is compared from multiple cameras are determined which is the best based on results and transmitting the video stream to the remote side.).
Wang, Hsin and Takahashi do not teach marks at least one of the master image and the slave image with a speaker label.
Duckworth teaches marks at least one of the master image and the slave image with a speaker label (see ¶ 0023. Cameras (202-205) can be labeled FR (far-right), CR (center right), CL (center left), and FL (far left). Different labels can also be used, for example, if camera 205 is used to capture images of all participants then it can be labeled as WR (whole room). Camera 202 can be a pan-tilt-zoom (PZT) type camera that captures the image of the current speaker only, from among the local participants, and thus can be labeled CS (current speaker).).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Wang, Hsin and Takahashi to incorporate label for a speaker. The modification provides for generating a label for the speaker for the camera.
Regarding claim 9, Wang teaches the video conference method according to claim 6, further comprising: receiving, by at least one of the master conference device and the slave conference device, voice data of a participant; recognizing an orientation of the voice data of the participant by at least one of the master conference device and the slave conference device, so as to generate master voice orientation data; comparing, by at least one of the master conference device and the slave conference device, the master voice orientation data with at least one relative angle formed between the speaker and at least one of the master conference device and the slave conference device; and determining, by at least one of the master conference device and the slave conference device (see fig. 1A-6, 10, ¶ 0023, 0029-0036, 0064-0066. The videoconference room picks best camera view in a videoconference room when multiple cameras are installed. One camera is designated as the primary camera and includes a microphone array. The microphone array is used to figure out where speech is coming from. The system then combines that audio direction with the primary camera’s image to identify who is speaking. It estimates the speaker’s pose, such as face direction and keypoints. The other cameras also analyze the people they see and estimate each person’s pose. Those poses are compared to the speaker’s pose so the same person can be identified across cameras. After that, the system chooses the camera that gives the most frontal view of the speaker. In some versions, the system can also create a composite view made from the best views of a participants. The conferencing system has a primary camera and slave cameras. The best image pose that is compared from multiple cameras are determined which is the best based on results and transmitting the video stream to the remote side.).
Wang, Hsin and Takahashi do not teach that the participant is the speaker and marking at least one of the master image and the slave image with a speaker label when the master voice orientation data matches the at least one relative angle.
Duckworth teaches that the participant is the speaker and marking at least one of the master image and the slave image with a speaker label when the master voice orientation data matches the at least one relative angle (see ¶ 0023. Cameras (202-205) can be labeled FR (far-right), CR (center right), CL (center left), and FL (far left). Different labels can also be used, for example, if camera 205 is used to capture images of all participants then it can be labeled as WR (whole room). Camera 202 can be a pan-tilt-zoom (PZT) type camera that captures the image of the current speaker only, from among the local participants, and thus can be labeled CS (current speaker).).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Wang, Hsin and Takahashi to incorporate label for a speaker. The modification provides for generating a label for the speaker for the camera.
5. Claim(s) 5, 10 are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US 2022/0408015) in view of Hsin et al. (US 2025/0157186) in further view of Takahashi et al. (US 2017/0127017) in further view of Pivnicka et al. (US 2022/0172734).
Regarding claim 5, Wang, Hsin and Takahashi do not teach the video conference system according to claim 1, wherein the master conference device further includes a master microphone, the slave conference device further includes a slave microphone, and the master conference device compares gain of audio data of the master microphone with gain of audio data of the slave microphone; wherein, when the gain of the audio data of the master microphone is greater than the gain of the audio data of the slave microphone, the master conference device transmits the audio data of the master microphone to the remote user device; and when the gain of the audio data of the slave microphone is greater than the gain of the audio data of the master microphone, the slave conference device transmits the audio data of the slave microphone to the remote user device.
Pivnicka teaches wherein the master conference device further includes a master microphone, the slave conference device further includes a slave microphone, and the master conference device compares gain of audio data of the master microphone with gain of audio data of the slave microphone; wherein, when the gain of the audio data of the master microphone is greater than the gain of the audio data of the slave microphone, the master conference device transmits the audio data of the master microphone to the remote user device; and when the gain of the audio data of the slave microphone is greater than the gain of the audio data of the master microphone, the slave conference device transmits the audio data of the slave microphone to the remote user device (see fig. 1, 3, ¶ 0022, 0028. The system is able via the processing control unit to compare the gain pickup from each microphone, analyze the voices from each microphone input device, and then determine whether each microphone is near a speaking individual. The system may be able to classify each microphone as a microphone near a speaking individual (speaking microphones), which could be done by assigning values to each microphone. This can be where a primary microphone and slaves microphones determine speaker location and the gain values compared to each microphone data. The audio signal is transmitted to a remote party during a conference session.).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Wang, Hsin and Takahashi to incorporate microphone gain monitoring among multiple microphones to determine location of speakers in a room. label for a speaker. The modification provides for determine the gain value for a microphone located near a speaker.
Regarding claim 10, Wang, Hsin and Takahashi do not teach the video conference method according to claim 6, further comprising: comparing, by the master conference device, gain of audio data of a master microphone with gain of audio data of a slave microphone; and transmitting, by the master conference device, the audio data of the master microphone to the remote user device when the gain of the audio data of the master microphone is greater than the gain of the audio data of the slave microphone, or transmitting, by the slave conference device, the audio data of the slave microphone to the remote user device when the gain of the audio data of the slave microphone is greater than the gain of the audio data of the master microphone.
Pivnicka teaches wherein the master conference device further includes a master microphone, the slave conference device further includes a slave microphone, and the master conference device compares gain of audio data of the master microphone with gain of audio data of the slave microphone; wherein, when the gain of the audio data of the master microphone is greater than the gain of the audio data of the slave microphone, the master conference device transmits the audio data of the master microphone to the remote user device; and when the gain of the audio data of the slave microphone is greater than the gain of the audio data of the master microphone, the slave conference device transmits the audio data of the slave microphone to the remote user device (see fig. 1, 3, ¶ 0022, 0028. The system is able via the processing control unit to compare the gain pickup from each microphone, analyze the voices from each microphone input device, and then determine whether each microphone is near a speaking individual. The system may be able to classify each microphone as a microphone near a speaking individual (speaking microphones), which could be done by assigning values to each microphone. This can be where a primary microphone and slaves microphones determine speaker location and the gain values compared to each microphone data. The audio signal is transmitted to a remote party during a conference session.).
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Wang, Hsin and Takahashi to incorporate microphone gain monitoring among multiple microphones to determine location of speakers in a room. label for a speaker. The modification provides for determine the gain value for a microphone located near a speaker.
Conclusion
6. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ASSAD MOHAMMED whose telephone number is (571)270-7253. The examiner can normally be reached 9:00AM-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 571-272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ASSAD MOHAMMED/Examiner, Art Unit 2691
/DUC NGUYEN/Supervisory Patent Examiner, Art Unit 2691
Prosecution Insights