DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yu (2023/0412656) in view of Ong et al (11,165,992).
Consider claims 1, 9 and 17, Yu teaches a teleconference service system, an image processing system and non-transitory recording medium storing program code which, when executed by one or more processors, causes the one or more processors to perform a method comprising: calculating a scale ratio of images transmitted from sites (par. 0071; 0096; “an initial aspect ratio for displaying a video stream obtained from a current participant device connected to a video conference is determined. The initial aspect ratio may be a same initial aspect ratio used to display video streams obtained from one or more other participant devices connected to the video conference. Alternatively, the initial aspect ratio may be determined specific to the video stream obtained from the current participant device or specific to the current participant device itself. The initial aspect ratio may be determined by sampling video and/or audio content captured at the current participant device and/or based on one or both of display capabilities or orientation information for the current participant device. In some cases, the same initial aspect ratio can be used for all participant devices having a same number of users. In some cases, the initial aspect ratio may be determined by a client application running at the current participant device and used to connect the current participant device to the video conference.”); scaling each image using the scale ratio which was calculated (par. 0096; “In some cases, the same initial aspect ratio can be used for all participant devices having a same number of users”; 0097; “In some cases, to display the first video stream and the second video stream using the initial aspect ratio, images of the first video stream and of the second video stream are cropped according to the initial aspect ratio”); and displaying each image that has been scaled at the scale ratio at a terminal (par. 0097; “the video stream is displayed within a GUI of the video conference using the initial aspect ratio. For example, the video stream may be displayed within a user interface tile of the GUI. The user interface tile may have a size corresponding to the initial aspect ratio”).
Yu does not explicitly suggest calculating a scaling ratio at which a difference in subject size between respective images of different subjects after scaling at the scaling ratio is smaller than a difference in subject size between respective images of the different subjects before scaling at the scaling ratio (i.e., calculating a scaling ratio such that the subject sizes (of different subjects) within the respective images become approximately the same). In the same field of endeavor, Ong et al teaches such arrangement to provide aesthetically pleasing layout arrangement (col. 3 lines 12-29; col. 5 lines 51-61; “The image scaling engine 316 may be used to generate scaled facial images 318, 320, and 322 for each participant. The faces of the scaled facial images 318, 320, and 322 may have the same general dimensions. As such, the faces of participants distant from the camera 203 appear to have the same size as the faces of participants closer to the camera 203. In certain embodiments, the image scaling engine 316 executes a super-resolution up-scaling of the facial images 306, 314, and 310 as part of the scaling operations”). Therefore, it would have been obvious to one of the ordinary skills in the art before the effective filing to incorporate the teaching of Ong et al into view of Yu for the purpose mentioned above.
Consider claims 2, 10 and 18, the combination teaches wherein the method further comprises: determining layout coordinates when the images are enlarged or reduced at the scale ratio which was calculated and arranged on a conference screen displayed by the terminal, wherein the displaying includes displaying each image at the scale ratio and at the layout coordinates on the conference screen at the terminal (par. 0061 of Yu; “A user interface tile may, but need not, be generally rectangular. The size of a user interface tile may depend on one or more factors including the view style set for the video conference GUI at a given time and whether the one or more conference participants represented by the user interface tile are active speakers at a given time. The view style for the video conference GUI, which may be uniformly configured for all conference participants by a host of the subject conference or which may be individually configured by each conference participant, may be one of a gallery view in which all user interface tiles are similarly or identically sized and arranged in a generally grid layout or a speaker view in which one or more user interface tiles for active speakers are enlarged and arranged in a center position of the video conference GUI while the user interface tiles for other conference participants are reduced in size and arranged near an edge of the video conference GUI”; par. 0071; “in such a case, for example, where a gallery view layout is used for the GUI of the video conference, a first user interface tile within which the video stream obtained from the first participant device 506 is displayed may be the same size as a second user interface tile within which the video stream obtained from the second participant device 508 is displayed. In another example, where a single conference participant is participating in the video conference from the first participant device 506 and two conference participants are participating in the video conference from the second participant device 508, a first initial aspect ratio (e.g., 3:4) may be used to display the video stream obtained from the first participant device 506 and a second initial aspect ratio (e.g., 4:3) may be used to display the video stream obtained from the second participant device 508”).
Consider claims 3, 11 and 19, the combination teaches wherein the method further comprises: obtaining the subject size by detecting a predetermined subject in one image transmitted from each site, or obtaining the subject size from a teleconference service system; calculating an average subject size of multiple subjects in one image sent from each site; and calculating an overall average size, which is an average of the average subject sizes for each site, wherein the calculating the scale ratio calculates the scale ratio which when applied to the average subject size results in the overall average size (par. 0074 of Yu; “the sampling may also be performed to determine where conference participants depicted within the obtained video streams are located within those video streams (e.g., centered or off-center) and/or sizes of those conference participants within the respective video streams (e.g., indicative of a distance between a given conference participant and a camera of their participant device). For example, region of interest processing can be performed against the video stream obtained from a participant device, such as using object detection, to determine a location of a human face within the video frames of the video stream, in which that location may be recognized as the location of a conference participant. The object detection output may further indicate a size of the region of interest corresponding to the human face, such as using a bounding box surrounding the human face. Where the size of the human face is determined to be small, the video stream may be cropped and zoomed into to cause the human face to be more visible and to minimize the inclusion of content irrelevant to the video conference. In another example, where the participant is detected to be off-center relative to the camera used to capture the video stream, the video stream may be cropped to cause the participant to appear centered. Where any zooming (e.g., magnification) is determined for a video stream, the conferencing software 504, via the initial aspect ratio determination tool 514 or otherwise, may transmit instructions to the subject participant device to cause the camera of that participant device to begin capturing the video stream using a higher resolution to allow for the zoomed-in video frames to remain at a high quality output”; where average sizes are based on the bounding box and the number of participants).
Consider claims 5 and 13, the combination teaches wherein the method further comprises: arranging each of the images enlarged or reduced by the scale ratio vertically or horizontally from an edge of the conference screen, and reducing each of the images to fit within a height or width of the conference screen, in a case that a fixed number of images cannot be arranged in the height or width of the conference screen (par. 0078; 0088 of Yu; “The event is something occurring or otherwise determined during the video conference which may be relevant to adjusting an aspect ratio used for displaying a video stream of one or more participant devices connected to the video conference, to ensure that the content most relevant to the video conference is output for display within the GUI of the video conference. Examples of such events include, but are not limited to, changes in a number of conference participants participating in the video conference from a given participant device, changes in a number of objects relevant to the video conference depicted within a video stream obtained from a given participant device, changes in the orientation of a participant device, and changes in media presented for display within a GUI of the video conference”; par. 0089; “Displaying the video stream within the GUI of the video conference using the adjusted aspect ratio results in a change in the amount of space within the GUI used to display the video stream. For example, a size of a user interface tile within which the video stream is displayed may be changed according to the adjusted aspect ratio. In another example, the space within the user interface file used to display the video stream may be changed according to the adjusted aspect ratio without changing the size of the user interface tile itself”).
Consider claims 6 and 14, the combination teaches wherein the method further comprises: calculating an average subject size of multiple subject sizes appearing in one image transmitted from each site, and calculating the ratio of the average subject size to the average size inputted on the terminal as the scale ratio (par. 0074 of Yu; “the sampling may also be performed to determine where conference participants depicted within the obtained video streams are located within those video streams (e.g., centered or off-center) and/or sizes of those conference participants within the respective video streams (e.g., indicative of a distance between a given conference participant and a camera of their participant device). For example, region of interest processing can be performed against the video stream obtained from a participant device, such as using object detection, to determine a location of a human face within the video frames of the video stream, in which that location may be recognized as the location of a conference participant. The object detection output may further indicate a size of the region of interest corresponding to the human face, such as using a bounding box surrounding the human face. Where the size of the human face is determined to be small, the video stream may be cropped and zoomed into to cause the human face to be more visible and to minimize the inclusion of content irrelevant to the video conference. In another example, where the participant is detected to be off-center relative to the camera used to capture the video stream, the video stream may be cropped to cause the participant to appear centered. Where any zooming (e.g., magnification) is determined for a video stream, the conferencing software 504, via the initial aspect ratio determination tool 514 or otherwise, may transmit instructions to the subject participant device to cause the camera of that participant device to begin capturing the video stream using a higher resolution to allow for the zoomed-in video frames to remain at a high quality output”).
Consider claims 7 and 15, the combination teaches wherein the method further comprises: requesting a teleconference service system to trim a circumscribed rectangle of a predetermined subject from each of the images which is enlarged or reduced, in a case that a notification is received that a conference screen is in a material display mode; enlarging or reducing each of the images at the scale ratio; and transmitting the circumscribed rectangle of the predetermined subject trimmed from each of the images to the terminal at each site (par. 0074 of Yu; “The object detection output may further indicate a size of the region of interest corresponding to the human face, such as using a bounding box surrounding the human face. Where the size of the human face is determined to be small, the video stream may be cropped and zoomed into to cause the human face to be more visible and to minimize the inclusion of content irrelevant to the video conference. In another example, where the participant is detected to be off-center relative to the camera used to capture the video stream, the video stream may be cropped to cause the participant to appear centered. Where any zooming (e.g., magnification) is determined for a video stream, the conferencing software 504, via the initial aspect ratio determination tool 514 or otherwise, may transmit instructions to the subject participant device to cause the camera of that participant device to begin capturing the video stream using a higher resolution to allow for the zoomed-in video frames to remain at a high quality output”).
Consider claims 8 and 16, the combination teaches wherein: the subject size is a face size of participants in the conference (par. 0074 of Yu; “the sampling may also be performed to determine where conference participants depicted within the obtained video streams are located within those video streams (e.g., centered or off-center) and/or sizes of those conference participants within the respective video streams (e.g., indicative of a distance between a given conference participant and a camera of their participant device). For example, region of interest processing can be performed against the video stream obtained from a participant device, such as using object detection, to determine a location of a human face within the video frames of the video stream, in which that location may be recognized as the location of a conference participant. The object detection output may further indicate a size of the region of interest corresponding to the human face, such as using a bounding box surrounding the human face. Where the size of the human face is determined to be small, the video stream may be cropped and zoomed into to cause the human face to be more visible and to minimize the inclusion of content irrelevant to the video conference”).
Allowable Subject Matter
Claims 4, 12 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any response to this action should be mailed to:
Mail Stop ____(explanation, e.g., Amendment or After-final, etc.) Commissioner for Patents
P.O. Box 1450
Alexandria, VA 22313-1450
Facsimile responses should be faxed to:
(571) 273-8300
Hand-delivered responses should be brought to:
Customer Service Window
Randolph Building
401 Dulany Street
Alexandria, VA 22314
Any inquiry concerning this communication or earlier communications from the examiner should be directed to QUOC DUC TRAN whose telephone number is (571)272-7511. The examiner can normally be reached Monday-Friday 8:30am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached on (571) 272-7503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Quoc D Tran/
Primary Examiner, Art Unit 2691
October 8, 2026
Prosecution Insights