DETAILED ACTION
This action is in response to the application filed 12/19/2024. Claims 1 – 10 are pending and have
been examined.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 1 is objected to because of the following informalities:
Claim 1 states “a participant selecting step of, by the web conferencing system, receiving a selection of two or more participants from among the participants whose images images have been received in the participant image receiving step”. The word “images” is repeated in a manner that is believed to be a typographical error.
Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the claim is directed to a “program for causing a computer to execute” a screen composing method. The claimed program, standing alone, is not a process, machine, manufacture, or composition of matter, but rather constitutes instructions for causing a computer to perform the recited method steps. Accordingly, the claimed subject matter does not fall within one of four statutory categories of invention under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 2, 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Vander Mey et al. (U.S. Patent No. 9,113,032, hereinafter “Vander Mey”) in view of Xu et al. (U.S. Pub. No. 2022/0303478, hereinafter “Xu”).
Regarding Claim 1, Vander Mey teaches
A screen composing method using a web conferencing system (see Vander Mey Abstract, method includes determining a participation rating for each of a plurality of participants in a video conference), the screen composing method comprising:
a material displaying step of, by the web conference system, displaying a material used for a web conference on a display unit (see Vander Mey Column 3, lines 1 – 5, a first computing device 2 that may execute one or more applications (e.g., video conference application 8) and engage in a video conference with one or more other computing devices 20-1 through 20-5);
a participant image receiving step of, by the web conferencing system, receiving a captured image of each of a plurality of participants participating in the web conference (see Vander Mey Figure 1, selection module is within video conference application and Column 9, lines 39 – 42, Selection module 6 may process video data, e.g., image data 38 received from one or more of computing devices 20-1 through 20-5);
a participant selecting step of, by the web conferencing system, receiving a selection of two or more participants from among the participants whose images images have been received in the participant image receiving step (see Vander Mey Column 9, lines 42 – 46, Selection module 6 may select a subset of participants from a plurality of participants in a video conference, wherein selection module 6 or video conference application 8 provides signals to instruct display device 4 to display image data associated with the selected set of participants); and
the images based on the image regions of the participants being displayed on the display unit on a basis of a predetermined display pattern in the selected participant displaying step (see Vander Mey Column 16, lines 63 – 67 and Column 17, lines 1 - 4, the first through third selected participants may have the top three participation ratings. In one example, if the currently speaking participant is not one of the three selected participants, the currently speaking participant may be displayed in a designated graphical portion of the display device 84. If one of the three selected participants begins speaking, image data associated with the previously speaking participant may remain displayed in the designated graphical portion, Column 16, lines 47 – 51, Computing device 80 may comprise a display device 84. As shown in FIG. 4, display device 84 may be virtually partitioned into five sections. Display device 84 may display image data associated with four participants of a video conference), wherein
the participants participating in the web conference include a first participant and a second participant (see Vander Mey Column 4, lines 32 – 34, real-time video communication between first computing device 2 and second through sixth computing devices 20-1 through 20-5),
Vander Mey does not expressively teach
a selected participant displaying step of, by the web conferencing system, extracting image regions of the participants for the two or more participants selected in the participant selecting step from captured images of the participants and displaying images based on the extracted image regions of the participants together with the material on the display unit,
the images based on the image regions of the participants being images of face parts of the participants, the face parts being extracted from the captured images of the participants,
when the first participant and the second participant are determined to have talked with each other, the web conferencing system displays an image based on an image region of the first participant and an image based on an image region of the second participant on the display unit in the selected participant displaying step.
However, Xu teaches
a selected participant displaying step of, by the web conferencing system, extracting image regions of the participants for the two or more participants selected in the participant selecting step from captured images of the participants and displaying images based on the extracted image regions of the participants together with the material on the display unit (see Xu Paragraph [0021], For each video frame of near-end video captured by the camera (46), the control module (20) selects a view region, and the control module (20) or the video module (40) crops the video frame to the view region, Paragraph [0022], Then, the network module (50) outputs the encoded audio and video to the remote endpoints (60) via the network (55) using any appropriate protocol),
the images based on the image regions of the participants being images of face parts of the participants, the face parts being extracted from the captured images of the participants (see Xu Paragraph [0016], The face detection information identifies a face size and a face position of one or more faces detected in the frame. The frame is cropped and scaled according to one or more crop and scale parameters using the face detection information to obtain a modified frame. The crop and scale parameter(s) may be based on the frames of the multiple video streams. Example crop and scale parameter(s) identify a target frame region enclosing the portion of the frame to be included in the modified frame. The crop and scale parameter(s) may further identify a target face region enclosing a face in the frame. The target frame region and the target face region may be defined relative to a template that indicates the proportion of the size of the target frame region relative to the size of the target face region),
when the first participant and the second participant are determined to have talked with each other (see Xu Paragraph [0021], the control module (20) may select an area of the video frame depicting a participant who is currently speaking as the view region), the web conferencing system displays an image based on an image region of the first participant and an image based on an image region of the second participant on the display unit in the selected participant displaying step (see Xu Paragraph [0021], the control module (20) may select an area of the video frame depicting a participant who is currently speaking as the view region, Paragraph [0062], the receiving endpoint (e.g., the video module of the receiving endpoint) selects the active speaker layout (524) based on a layout rule indicating that the active speaker layout (524) is used when the number of input frames is at least 3. The layout rules corresponding to the active speaker layout (524) assign a small frame size (526) to the frames containing 1 face, and a large frame size (528) to a frame containing more than 1 face).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a web conference screen composition method that allows users to select a plurality of participants and automatically arranges their video images in predetermined regions or patterns alongside the conference materials (as taught in Vander Mey), with a system that extracts face-image regions from captured images of participants and displays those regions with conference materials in which when a first and second participant are have determined to speak, their respective face-image regions are displayed (as taught in Xu), the motivation being to provide a consistent and organized conference interface by extracting relevant participant face regions and arranging them in a predetermined layout (see Xu Paragraph [0001]).
Regarding Claim 2, Vander Mey in view of Xu teaches
The screen composing method using the web conferencing system according to claim 1, further comprising
a speaker identifying step of, by the web conferencing system, identifying a person who has spoken, out of the participants participating in the web conferencing (see Vander Mey Column 7, lines 20 – 21, a conversation property may include an indication that a participant is currently speaking in a video conference, Column 10, lines 59 – 63, One example conversation property may include whether a participant is currently speaking at a particular moment during the video conference. When a participant is speaking, selection module 6 may indicate the presence of this property, or assign a value (such as a positive value) to the property), wherein
the participant selecting step further includes a step of selecting two or more participants from among the participants, using information about the identified person who has spoken (see Vander Mey Column 16, lines 63 – 67 and Column 17, lines 1 - 4, the first through third selected participants may have the top three participation ratings. In one example, if the currently speaking participant is not one of the three selected participants, the currently speaking participant may be displayed in a designated graphical portion of the display device 84. If one of the three selected participants begins speaking, image data associated with the previously speaking participant may remain displayed in the designated graphical portion, Column 10, lines 64 – 66, any currently speaking participant may be given a highest rating or ranking of the plurality of participants. In another example, a weight may be assigned to the conversation property).
Regarding Claim 9, it is rejected similarly as Claim 1. The program can be found in Vander Mey (Column 8, line 19, program instructions).
Regarding Claim 10, Vander Mey in view of Xu teaches
A non-transitory computer-readable information recording medium storing the program according to claim 9 (see Vander Mey Column 8, lines 8 – 9, non-transitory computer-readable storage medium).
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Vander Mey et al. (U.S. Patent No. 9,113,032, hereinafter “Vander Mey”) in view of Xu et al. (U.S. Pub. No. 2022/0303478, hereinafter “Xu”) and Chen et al. (U.S. Pub. No. 2023/0237838, hereinafter “Chen”).
Regarding Claim 3, Vander Mey in view of Xu teaches all the limitations of claim 1, but does not expressively teach
The screen composing method using the web conferencing system according to claim 1, wherein
the first participant and the second participant include a presenter and a questioner, and
when the questioner is determined to have asked a question about an explanation by the presenter, the web conferencing system displays an image of a video region of the presenter and an image based on a video region of the questioner on the display unit in the selected participant displaying step.
However, Chen teaches
The screen composing method using the web conferencing system according to claim 1, wherein
the first participant and the second participant include a presenter and a questioner (see Chen Figure 7, displaying a presenter and a questioner in main video output regions, Paragraph [0044], In step S176, the main display area (the main display area MA as shown in FIG. 5) is decomposed into a first split area (the first split area SC1 as shown in FIG. 5) and a second split area (the second split area SC2 as shown in FIG. 5) and a question-and-answer mode is started, as shown in FIG. 7, Paragraph [0045], In step S178, in the question-and-answer mode, the first human face frame image 212 is configured to the first split area SC1, and the second human face frame image 232 is configured to the second split area SC2), and
when the questioner is determined to have asked a question about an explanation by the presenter, the web conferencing system displays an image of a video region of the presenter and an image based on a video region of the questioner on the display unit in the selected participant displaying step (see Chen Paragraph [0043], the sound station 106 can receive a sound signal from the person corresponding to the first identity label. The sound signal includes contents of the second identity label (such as the name or job position of the person who has been named) and a keyword to start a question-and-answer mode, Paragraph [0044], In step S176, the main display area (the main display area MA as shown in FIG. 5) is decomposed into a first split area (the first split area SC1 as shown in FIG. 5) and a second split area (the second split area SC2 as shown in FIG. 5) and a question-and-answer mode is started, as shown in FIG. 7, Paragraph [0045], In step S178, in the question-and-answer mode, the first human face frame image 212 is configured to the first split area SC1, and the second human face frame image 232 is configured to the second split area SC2).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a web conferencing system that selects participants, extracts their face-image regions from captured participant images, and automatically displays those regions alongside conference materials according to a predetermined display layout, including displaying the image regions of participants when they are determined to have talked (as taught in Vander Mey in view of Xu), with a video conferencing system that displays video regions of a presenter and a questioner when the questioner is determined to have asked the presenter a question (as taught in Chen), the motivation being to improve conversation context and clarity by automatically displaying a presenter alongside a questioner when a question is asked, making it immediately clear who is speaking and who is being addressed (see Chen Paragraphs [0003] and [0004]).
Claims 4 and 5 are rejected under 35 U.S.C. 103 as being unpatentable over Vander Mey et al. (U.S. Patent No. 9,113,032, hereinafter “Vander Mey”) in view of Xu et al. (U.S. Pub. No. 2022/0303478, hereinafter “Xu”) and Shanmukhadas et al. (U.S. Pub. No. 2012/0274730, hereinafter “Shanmukhadas”).
Regarding Claim 4, Vander Mey in view of Xu teaches all the limitations of claim 1, but does not expressively teach
The screen composing method using the web conferencing system according to claim 1, further comprising a display image storing step of, by the web conferencing system, storing a display image including the material and the images based on the image regions of the participants displayed on the display unit in the selected participant displaying step.
However, Shanmukhadas teaches
The screen composing method using the web conferencing system according to claim 1, further comprising a display image storing step of, by the web conferencing system, storing a display image including the material and the images based on the image regions of the participants displayed on the display unit in the selected participant displaying step (see Shanmukhadas Paragraph [0113], In 1310, a recording of the videoconference may be provided to one or more clients. The recording of the videoconference may supply the first videoconference information, the second videoconference information, and/or the timeline of the second videoconference information in a single stream or package or may provide them separately as desired, Paragraph [0096], the recording server may record or store the videoconference based on the selected configurations).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a web conferencing system that selects participants, extracts their face-image regions from captured participant images, and automatically displays those regions alongside conference materials according to a predetermined display layout, including displaying the image regions of participants when they are determined to have talked (as taught in Vander Mey in view of Xu), with a system that stores a display image that includes the conference material of a video conference and participant images shown on the display (as taught in Shanmukhadas), the motivation being to allow the system to capture and preserve a record of a conference for later reference or review (see Shanmukhadas Paragraph [0115]).
Regarding Claim 5, Vander Mey in view of Xu and Shanmukhadas teaches
The screen composing method using the web conferencing system according to claim 4, further comprising a recorded information revising step of revising an image pertaining to the material in the display image stored in the display image storing step and storing the revised image in a case where the material has been revised (see Shanmukhadas Paragraph [0072], When composited images are sent to the recording server, the endpoint may indicate the boundaries of the composite video image so that the layout may be changed at a later point, e.g., by streaming clients).
Claims 6, 7 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over Vander Mey et al. (U.S. Patent No. 9,113,032, hereinafter “Vander Mey”) in view of Xu et al. (U.S. Pub. No. 2022/0303478, hereinafter “Xu”) and Szigeti et al. (U.S. Pub. No. 2023/0297556, hereinafter “Szigeti”).
Regarding Claim 6, Vander Mey in view of Xu teaches all the limitations of claim 1, but does not expressively teach
The screen composing method using the web conferencing system according to claim 1, further comprising a code information displaying step of, by the web conferencing system, displaying code information on the display unit.
However, Szigeti teaches
The screen composing method using the web conferencing system according to claim 1, further comprising a code information displaying step of, by the web conferencing system, displaying code information on the display unit (see Szigeti Paragraph [0055], FIGS. 7A and 7B depict two-dimensional barcodes 700 and 750, in accordance with example embodiments. FIG. 7A depicts a two-dimensional barcode 700, which is used to encode a key value that can be used to retrieve details relating to content item (e.g., the identity of the content author, the existence of updates, etc.). The two-dimensional barcode may be a matrix barcode such as a quick response (QR) code or other such format, Paragraph [0056], barcode 750 may be graphically inserted into or adjacent to a content item so that barcode 750 is detectable, Paragraph [0018], copies of content items obtained from video recordings of a previous presentation, to be tracked, Paragraph [0019], enabling both tracking and updating of content items, including content items that are shared via collaboration applications (e.g., video presentation applications), Paragraph [0038], Session hosting module 126 may initiate collaboration sessions (e.g., video conferences, etc.) between participants so that one or more users can share data, including video data, audio data, and/or data corresponding to content items, with the other participants).
It would have been obvious to one of ordinary skill in the art before the effective filing date of
the claimed invention to combine the teaching of a web conferencing system that selects participants, extracts their face-image regions from captured participant images, and automatically displays those regions alongside conference materials according to a predetermined display layout, including displaying the image regions of participants when they are determined to have talked (as taught in Vander Mey in view of Xu), with displaying code information on a video conference display (as taught in Szigeti), the motivation being to provide a data management method that enables tracking of content to observe applicable data and also ensuring that copied content can be updated (see Szigeti Paragraph [0002]).
Regarding Claim 7, Vander Mey in view of Xu and Szigeti teaches
The screen composing method using the web conferencing system according to claim 6, wherein the material includes a plurality of pages, and in the code information displaying step, for each page displayed on the display unit, code information corresponding to the page is displayed on the display unit (see Szigeti Paragraph [0053], FIG. 6 depicts a view of a user interface 600 including two-dimensional barcode locations, in accordance with an example embodiment. As depicted, user interface 600 shows an example of a slide of a presentation or page of a document in which a content item 610 is present, Paragraph [0054], Content items can correspond to pages of a document, paragraphs of a document, particular in-line images or graphs, slides in a slideshow presentation, particular images, graphs, or other media in a particular slide of a slideshow presentation, and the like).
Regarding Claim 8, Vander Mey in view of Xu and Szigeti teaches
The screen composing method using the web conferencing system according to claim 6, wherein as the web conference progresses, the web conferencing system updates the code information to obtain updated code information, and the web conferencing system displays the updated code information on the display unit (see Szigeti Abstract, In response to determining that the author has updated the content item, the user is notified that an updated version of the content item is available for presentation in the communication session, Paragraph [0018], Accordingly, present embodiments provide a central content tracking and/or updating solution in which content items within documents can be managed at any level of granularity in order to track usage of those content items, to identify when outdated content items are being used, and/or to provide automated approaches to updating content items that have been copied into other documents and/or are being presented. Furthermore, a server that hosts a collaborative application session may detect keys of content items that a user desires to share during the session, and can identify when presented content items are out-of-date and automatically alert the author or presenter, and/or update the content item).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of References Cited for a listing of analogous art.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CARISSA A JONES whose telephone number is (703)756-1677. The examiner can normally be reached Telework M-F 6:30 AM - 4:00 PM CT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Duc Nguyen can be reached at 5712727503. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CARISSA A JONES/Examiner, Art Unit 2691
/DUC NGUYEN/Supervisory Patent Examiner, Art Unit 2691
