DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1, 10, and 17 are amended.
Claims 1-20 are pending.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 2, 9-11, and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US PGPUB #2022/0270216) in view of Luks et al. (US PGPUB # 2018/0063480).
Regarding Claim 1, Wang discloses an information handling system (title, abstract, figs. 1-13) comprising:
a processer operable to execute instructions to process information (Wang ¶0021 discloses control module 20 include software executable CPU. ¶0027: processing unit 110; fig. 2);
a memory interfaced with the processor and operable to store the instructions and information (Wang ¶0027: memory 140; fig. 2);
a network interface controller interfaced with the processor and operable to communicate the information through a network (Wang ¶0021: network module 50);
PNG
media_image1.png
1276
1849
media_image1.png
Greyscale
a camera operable to capture visual images within a field of view (Wang camera 46; ¶0024 discloses the camera 46 includes a wide-angle lens. Due to the nature of wide-angle lenses, video [and still images] captured by the camera 46 includes both distortion [405] and deformation [507] effects); and
a non-transitory memory storing instructions that when executed (Wang ¶0045) cause:
communication of the visual images of the field of view through the network, the visual images including plural individuals (Wang ¶0030 discloses the videoconferencing endpoint 10 captures frames [705, fig. 7] of video, selectively crops the frames [705] to view regions, and selectively applies deformation [507. fig. 5] or distortion-correction [508] to the view regions based on size of the view regions. Because distortion [405] can be more noticeable in relatively larger view regions and deformation [507] can be more noticeable in relatively smaller view regions, selectively using one of the correction techniques enhances quality of video during a communication session by addressing irregularities that can be more noticeable to a communication session participant. Thus, fig. 2 illustrates an example physical configuration of a device that selectively corrects deformation [507] or distortion [405] to enhance quality of a video);
detection of the plural individuals in the field of view (Wang fig. 4: 409, 410, 411, 412);
for each of the plural individuals:
determination of location in a predetermined inner angular range of the field of view (Wang fig. 4: 301) or a predetermined outer angular range of the field of view (Wang fig. 4);
PNG
media_image2.png
930
1193
media_image2.png
Greyscale
PNG
media_image3.png
801
1128
media_image3.png
Greyscale
when in the inner angular range, cropping the individual for communication as a gallery window (Wang fig. 3: 302; fig. 5: 511; ¶0031 discloses in many artificial intelligence-based cameras 46, when an active speaker is determined an active speaker view is formed by cropping the region containing the speaker's face [308] from a full view captured by the camera 46. When someone at the far end looks at the feed containing the wide view and the feed containing the active talker, he or she will tend to notice [and be distracted by] distortion [405] in the wide view, while he or she will tend to notice [and be distracted and/or disturbed by] any deformation [507] of the face [308] if present. In a full view, the viewer would care more about the geometric reality of background since it occupies the most regions in an image. In active speaker view, the viewer cares more about proper depiction of the single person who is the subject of the active speaker view); [[and]]
when in the outer angular range, cropping the individual and correcting the cropping for perspective distortion for communication as a gallery window (Wang fig. 4: 409; fig. 5: 409; ¶0030 discloses the videoconferencing endpoint 10 captures frames [705: fig. 7] of video, selectively crops the frames [705] to view regions, and selectively applies deformation [507: fig. 5] or distortion-correction [508] to the view regions based on size of the view regions. Because distortion [405: fig. 4] may be more noticeable in relatively larger view regions and deformation [507] may be more noticeable in relatively smaller view regions, selectively using one of the correction techniques enhances quality of video during a communication session by addressing irregularities that may be more noticeable to a communication session participant. Thus, fig. 2 illustrates an example physical configuration of a device that selectively corrects deformation [507] or distortion [405] to enhance quality of a video).
Wang may not explicitly disclose communication of a gallery having each of the gallery windows, the gallery superimposed over the visual images.
However, Luks (title, abstract, figs. 1-8) teaches communication of a gallery having each of the gallery windows, the gallery superimposed over the visual images (Luks ¶0023 discloses the online meeting service 114 can be configured to determine attributes of various meeting attendants and display video feeds from the attendants or representative icons [e.g., pictures; based on cropping] according to predefined, customizable, or dynamic configurations. For example, dominant speakers can be defined based on who is speaking, who is an important attendee, who is presenting, and similar factors. Furthermore, whether or not panoramic views are available may determine a layout of an online meeting user interface. Moreover, content such as presented information, chat sessions, and comparable items may be laid out in conjunction with the people being displayed on the user interface. In some embodiments, the people may be displayed in a gallery view. ¶0042 discloses example user interface 350 of fig. 3B shows a gallery view with two groups of attendants displayed as active people 354 and inactive people 355 over canvas 352. The row of active people 354 may also be referred to as "standing row" and the row of inactive people 355 may also be referred to as "sitting row". As active speakers change during the meeting, replacements may be made automatically between the two rows. As in example user interface 340, content 354 and user 356 representations may be optionally displayed to allow the user to be able to switch between different views).
Wang and Luks are analogous art as they pertain to video conference. Therefore it would have been obvious to someone of ordinary skill in the art before the effective filing date of the invention was made to modify cropping the frames (as taught by Wang) to allow gallery views in online conference user interfaces, by determining an optimized presentation of attendees and other information in an online meeting user interface (as taught by Luks, ¶0024) for increased efficiency, reduced local and cloud computing resource usage, reduced bandwidth usage, and increased security and collaboration which can be achieved through avoidance of usage of multiple user interfaces, applications, and network channels (Luks, ¶0024).
Regarding Claim 2, Wang in view of Luks discloses the information handling system of claim 1 further comprising: instructions stored in the non-transitory memory that cause:
determination of a distance to each individual in the outer angular range (Wang ¶0033 discloses fig. 3 illustrates an image 306, which has been corrected [508] for distortion [405]. Correction [508] for distortion [405] is evidenced by the fact that the lines of the furniture and windows are straight. Within the room view [401] depicted in the image 306, we see Xingyue, who is the subject 302 of the image 306. Xingyue's face 308 is depicted within a portion 309 of the image 306); and
when the distance is greater than a predetermined amount, communication of the gallery window without correcting the perspective distortion (Wang ¶0033 discloses the portion 309 of the image 306 depicting Xingyue's face 308 has a center 312. The depiction of Xingyue's face 308 shows some deformation [507]. Image deformation [507] of a subject 302 increases as the distance 300 of the subject 302 from the center 304 of the image 306 increases. Image deformation [507] of a subject 302 increases as the distance 300 of the center 312 of the portion 309 of the image 306 containing the subject 302 from the center 304 of the image 306 increases. The center 304 of the image 306 defines and falls within the central region 301 of the image 306. In at least one example of this disclosure, for a magnified face 308 a correction method which relies more heavily on deformation-reduction [1050] than would be desirable can be used); and
when the distance is less than a predetermined amount, correction of the perspective distortion before communication of the gallery window (Wang ¶0037 discloses whether a facial image is located in the minimal-deformation zone is thus a major determiner for whether distortion-correction 508 will induce deformation 507, such as occurred for Hailin in fig. 5. Another major determiner is the distance of the image's 400 subject from the camera 46. This principle is illustrated in fig. 6. In fig. 6, image 602 and image 604 have both undergone lens distortion-correction 508. In image 602, Tianran 603 is about 0.5 meters from the camera 46. In image 604, Tianran 603 is about 2.0 meters from the camera 46. In image 606, Tianran 603 from image 602 is magnified in a zoom [active talker] view. In image 608, Tianran 603 from image 604 is magnified. When Tianran 603 is sitting closer to the camera 46 [e.g., 602, 606], lens distortion-correction 508 causes more deformation 507 than when Tianran 603 sits farther away [e.g., 604, 608]. The cause of this is illustrated in fig. 7).
Regarding Claim 9, Wang in view of Luks discloses the information handling system of claim 2 wherein the instructions further:
detect a speaker is one of the individuals in the gallery having a corrected perspective distortion (Wang ¶0033 discloses the portion 309 of the image 306 depicting Xingyue's face 308 has a center 312. The depiction of Xingyue's face 308 shows some deformation [507]. Image deformation [507] of a subject 302 increases as the distance 300 of the subject 302 from the center 304 of the image 306 increases. Image deformation [507] of a subject 302 increases as the distance 300 of the center 312 of the portion 309 of the image 306 containing the subject 302 from the center 304 of the image 306 increases. The center 304 of the image 306 defines and falls within the central region 301 of the image 306. In at least one example of this disclosure, for a magnified face 308 a correction method which relies more heavily on deformation-reduction [1050] than would be desirable can be used); and
present the cropping of the speaker with the corrected perspective distortion in a speaker window (Wang ¶0030 discloses the videoconferencing endpoint 10 captures frames [705: fig. 7] of video, selectively crops the frames [705] to view regions, and selectively applies deformation [507: fig. 5] or distortion-correction [508] to the view regions based on size of the view regions. Because distortion [405: fig. 4] may be more noticeable in relatively larger view regions and deformation [507] may be more noticeable in relatively smaller view regions, selectively using one of the correction techniques enhances quality of video during a communication session by addressing irregularities that may be more noticeable to a communication session participant. Thus, fig. 2 illustrates an example physical configuration of a device that selectively corrects deformation [507] or distortion [405] to enhance quality of a video).
Claims 10-11 and 17-18 are rejected for the same reasons as set forth in Claims 1 and 2.
Claims 3-8, 12-16, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wang et al. (US PGPUB #2022/0270216) in view of Luks et al. (US PGPUB # 2018/0063480) further in view of Zhou et al. (US PGPUB #2022/0335579).
Regarding Claim 3, Wang in view of Luks discloses the information handling system of claim 2 but may not explicitly disclose wherein the perspective distortion correction comprises scaling a trapezoid bounding box defined around the individual to a rectangle shape.
However, Zhou (title, abstract, figs. 1-7) teaches wherein the perspective distortion correction comprises scaling a trapezoid bounding box defined around the individual to a rectangle shape (Zhou ¶0098 discloses fig. 6 illustrates an exemplary wide FOV image 100 and a framed and distortion-corrected portion thereof 605 comprising two ROIs [610A/610B] composited side-by-side, according to one or more embodiments. As in the example described above with reference to fig. 1, the process 600 illustrated in fig. 6 identifies the same two first ROIs 125A and 125B in the image 100. However, rather than determining a second ROI 130' that contains two first ROIs 125A and 125B and using that second ROI 130' to determine a final portion for use in the generation of the output image version of image 100 [as in fig. 1], the process 600 illustrated in fig. 6 elects instead to obtain and perspective correct each of the two first ROIs 125A and 125B independently, e.g., according to the process described in fig. 5A, resulting in perspective-corrected versions of the human subjects 105" and 110" appearing in regions 610A and 610B, respectively, of output image 605).
Wang, Luks, and Zhou are analogous art as they pertain to distortion correction in video conference. Therefore it would have been obvious to someone of ordinary skill in the art before the effective filing date of the invention was made to modify the teachings of Wang in view of Luks in light of the teachings of Zhou to obtain an incoming image stream from one or more image capture devices for applying perspective distortion correction (as taught by Zhou, ¶0083) for providing improved image framing and perspective distortion correction for wide FOV video image streams (Zhou, ¶0008).
Regarding Claim 4, Wang in view of Luks and Zhou discloses the information handling system of claim 3
wherein the instructions further comprise a look up table storing plural angles in the outer angular range, each of the plural angles associated with a scaling factor for scaling the trapezoid bounding box (Wang fig. 11: chart 1104; ¶0043 and ¶0053: lookup table).
Regarding Claim 5, Wang in view of Luks and Zhou discloses the information handling system of claim 4
wherein inner angular range is 80 degrees (Wang fig. 11; ¶0043 discloses a field of view 1102 for a wide-angle camera 46 with a 160-degree lens [inner angular range of 80 degrees can be obvious design choice]. The blind regions 1108 define the limits of the camera's field of view 1102. The camera's 46 field of view 1102 has a central region 1106 [e.g., 301]. The field of view 1102 can be subdivided into zones [A, B, C, D, E, F] according to distance from the camera 46 and angular distance away from the central region 1106. Each of the zones [A, B, C, D, E, F] can have its own lookup table as organized in chart 1104).
Regarding Claim 6, Wang in view of Luks and Zhou discloses the information handling system of claim 5
wherein the distance predetermined amount is two meters (Wang ¶0037 discloses in image 604, Tianran 603 is about 2.0 meters from the camera 46).
Regarding Claim 7, Wang in view of Luks discloses the information handling system of claim 3 but may not explicitly disclose wherein the trapezoid bounding box scaling comprises scaling up the bounding box inner angle scale to the bounding box outer angle scale.
However, Zhou (title, abstract, figs. 1-7) teaches wherein the trapezoid bounding box scaling comprises scaling up the bounding box inner angle scale to the bounding box outer angle scale (Zhou fig. 5E; fig. 5F: 574).
Wang, Luks, and Zhou are analogous art as they pertain to distortion correction in video conference. Therefore it would have been obvious to someone of ordinary skill in the art before the effective filing date of the invention was made to modify the teachings of Wang in view of Luks in light of the teachings of Zhou to obtain an incoming image stream from one or more image capture devices for applying perspective distortion correction (as taught by Zhou, ¶0083) for providing improved image framing and perspective distortion correction for wide FOV video image streams (Zhou, ¶0008).
Regarding Claim 8, Wang in view of Luks and Zhou discloses the information handling system of claim 7. But Wang may not explicitly disclose wherein the scale is the inverse of the cosine of the angle.
However, Zhou (title, abstract, figs. 1-7) teaches wherein the scale is the inverse of the cosine of the angle (Zhou fig. 5E; fig. 5F: 574. Scaling to be inverse of the cosine of the angle can be a design choice).
Wang, Luks, and Zhou are analogous art as they pertain to distortion correction in video conference. Therefore it would have been obvious to someone of ordinary skill in the art before the effective filing date of the invention was made to modify the teachings of Wang in view of Luks in light of the teachings of Zhou to obtain an incoming image stream from one or more image capture devices for applying perspective distortion correction (as taught by Zhou, ¶0083) for providing improved image framing and perspective distortion correction for wide FOV video image streams (Zhou, ¶0008).
Regarding Claim 16, Wang in view of Luks discloses the information handling system of claim 11 but may not explicitly disclose wherein the cropping the individual and correcting the cropping for perspective distortion are performed with an image sensor processing resource of a camera.
However, Zhou (title, abstract, figs. 1-7) teaches wherein the cropping the individual and correcting the cropping for perspective distortion are performed with an image sensor processing resource of a camera (Zhou figs. 1-2B, 5A-5B, 5E-6; ¶0098 discloses fig. 6 illustrates an exemplary wide FOV image 100 and a framed and distortion-corrected portion thereof 605 comprising two ROIs [610A/610B] composited side-by-side, according to one or more embodiments. As in the example described above with reference to fig. 1, the process 600 illustrated in fig. 6 identifies the same two first ROIs 125A and 125B in the image 100. However, rather than determining a second ROI 130' that contains two first ROIs 125A and 125B and using that second ROI 130' to determine a final portion for use in the generation of the output image version of image 100 [as in fig. 1], the process 600 illustrated in fig. 6 elects instead to obtain and perspective correct each of the two first ROIs 125A and 125B independently, e.g., according to the process described in fig. 5A, resulting in perspective-corrected versions of the human subjects 105" and 110" appearing in regions 610A and 610B, respectively, of output image 605).
Wang, Luks, and Zhou are analogous art as they pertain to distortion correction in video conference. Therefore it would have been obvious to someone of ordinary skill in the art before the effective filing date of the invention was made to modify the teachings of Wang in view of Luks in light of the teachings of Zhou to obtain an incoming image stream from one or more image capture devices for applying perspective distortion correction (as taught by Zhou, ¶0083) for providing improved image framing and perspective distortion correction for wide FOV video image streams (Zhou, ¶0008).
Claims 12-15 and 19 are rejected for the same reasons as set forth in Claims 3-8.
Regarding Claim 20, Wang in view of Luks and Zhou discloses the videoconference system of claim 19 wherein the instructions further:
detect a speaker is one of the individuals in the gallery having a corrected perspective distortion (Wang ¶0033 discloses the portion 309 of the image 306 depicting Xingyue's face 308 has a center 312. The depiction of Xingyue's face 308 shows some deformation [507]. Image deformation [507] of a subject 302 increases as the distance 300 of the subject 302 from the center 304 of the image 306 increases. Image deformation [507] of a subject 302 increases as the distance 300 of the center 312 of the portion 309 of the image 306 containing the subject 302 from the center 304 of the image 306 increases. The center 304 of the image 306 defines and falls within the central region 301 of the image 306. In at least one example of this disclosure, for a magnified face 308 a correction method which relies more heavily on deformation-reduction [1050] than would be desirable can be used); and
present the cropping of the speaker with the corrected perspective distortion in a speaker window (Wang ¶0030 discloses the videoconferencing endpoint 10 captures frames [705: fig. 7] of video, selectively crops the frames [705] to view regions, and selectively applies deformation [507: fig. 5] or distortion-correction [508] to the view regions based on size of the view regions. Because distortion [405: fig. 4] may be more noticeable in relatively larger view regions and deformation [507] may be more noticeable in relatively smaller view regions, selectively using one of the correction techniques enhances quality of video during a communication session by addressing irregularities that may be more noticeable to a communication session participant. Thus, fig. 2 illustrates an example physical configuration of a device that selectively corrects deformation [507] or distortion [405] to enhance quality of a video).
Response to Arguments
Applicant’s arguments with respect to claim(s) 1, 10, and 17 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YOGESHKUMAR G PATEL whose telephone number is (571)272-3957. The examiner can normally be reached 7:30 AM-4 PM PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at (571) 272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YOGESHKUMAR PATEL/Primary Examiner, Art Unit 2691
Prosecution Insights