DETAILED ACTION
This action is responsive to the response filed on 1/20/2026.
Claims 1, 3-8, 10-14, 16, 17, 21, and 24-28 are now pending in this application. Claims 1, 3-8, 11-14, 17, and 21 have been amended. Claims 18-20 and 22-23 have been cancelled. Claims 24-28 have been added. Claims 1, 8, and 14 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1, 8, 14 are objected to because of the following informalities:
Claims 1, 8, and 14, replace … the one or more outputs comprising an indication of a level of confidence that the one or more pixels of the image data correspond to the first participant… with … the one or more outputs comprising an indication of a level of confidence that for proper antecedent basis.
Claims 5 and 12, replace … from the image data, a set of pixels corresponding to the first participant … with … from the image data, the set of one or more pixels corresponding to the first participant … for proper antecedent basis.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1, 3-8, 10-14, 16, 17, 21, and 24-28 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Each of independent claims 1, 8, and 14 recites “responsive to determining that the level of confidence satisfies the confidence criterion, generating a view of the electronic document comprising an image portion composed of the one or more pixels and depicting the first participant in at least one region of the electronic document with at least one of the first content or the second content”.
Examiner notes that the specifications, as originally disclosed, do not disclose that the generation of the electronic document as recited is responsive to the determination that the level of confidence satisfies the confidence criterion. The specifications merely describe in [0056]-[0057] that the identification of the one or more pixels and the generation of the image portion depicting the first participant composed of the identified one or more pixels are responsive to determining that the level of confidence of these pixels satisfies the confidence criterion. Thus, this aspect has been determined to raise new matter issues. The dependent claims do not recite additional limitations to resolve this issue.
For purposes of art rejection, Examiner suggests replacing the above-indicated limitation of each of the independent claims with:
“responsive to determining that the level of confidence satisfies the confidence criterion, generating a an image portion composed of the one or more pixels and depicting the first participant;
generating a view of the electronic document comprising the image portion in at least one region of the electronic document with at least one of the first content or the second content.”
Appropriate correction is required.
Additionally, each of dependent claims 4, 11, and 17 recites “the one or more outputs of the machine learning model further comprise an indication of one or more regions of the electronic document that satisfy an image placement criterion, the one or more regions comprising the at least one region”.
Examiner notes that the specifications, as originally disclosed, do not disclose that the indication of one or more regions of the electronic document that satisfy an image placement criterion is an output of the machine learning model. The specifications merely describe a document region identifier that can identify one or more regions of the electronic document that satisfy an image placement criterion. Thus, this aspect has been determined to raise new matter issues.
For purposes of art rejection, Examiner suggests replacing the above-indicated limitation in each of dependent claims 4, 11, and 17 with:
“generating the view of the electronic document comprises generating an indication of one or more regions of the electronic document that satisfy an image placement criterion, the one or more regions comprising the at least one region”.
Appropriate correction is required.
Additionally, claim 24 recites “updating the generated view to comprise the image portion depicting the second participant and the second content of the electronic document”.
Examiner notes that the specifications, as originally disclosed, merely disclose updating the generated view to comprise the image portion depicting the first participant (the presenter) and the second content of the electronic document.. Thus, this aspect has been determined to raise new matter issues.
For purposes of art rejection, Examiner suggests replacing the above-indicated limitation in claim 24 with:
“updating the generated view to comprise the image portion depicting the first participant and the second content of the electronic document”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-5, 7, 8, 10-12, 14, 16, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Sharp et al., US PGPUB 2022/0191258 A1 (hereinafter as Sharp) in view of Nguyen et al., US Patent No. 9,881,207 Bl (hereinafter as Nguyen).
Regarding independent claim 1, Sharp teaches a method [see e.g. [0016]] performed by a set of processing devices [see e.g. the one or more processors 202 in fig. 2], the method comprising:
receiving a request to initiate a document sharing operation to share an electronic document displayed via a first graphical user interface (GUI) on a first client device associated with a first participant of a conference call with a second participant of the conference call via a second GUI on a second client device [see e.g. [0016] and [0020] both indicating an attendee selection that results in sharing presentation content from the attendee’s device; see also the attendee’s choice to be a presenter of content such as a document as described in [0019]], wherein the electronic document comprises first content and second content [note in [0072] the first and second pages or slides of a document being presented];
receiving at least image data comprising an image depicting an environment of the first participant during the conference call [note e.g. in fig. 6, step 620, receiving video streams from individual meeting endpoints ((which includes image data corresponding to a view of the first participant as captured by a camera device, which would include a capture of the entire view, part of which is the participant and the remaining part is the surroundings); see also the description in [0049] and [0055]];
obtaining an image portion (extracted from the image data) depicting the first participant [see e.g. the last 2 sentences in [0055] indicating obtaining specific video images depicting the attendee (especially note the inclusion of the attendee’s face) based on the video stream data (image data) by using facial and/or body recognition software and cropping the video content];
generating a view of the electronic document comprising the image portion depicting the first participant in at least one region of the electronic document with at least one of the first content or the second content and providing, during the conference call and for presentation via the second GUI, the generated view of the electronic document [note e.g. in fig. 6, steps 640 and 650 indicating the integration of the video content (including the image depicting the first participant as per the portions cited above) within the shared document and providing them for presentation via a remote GUI on a remote client device; again see also the description in [0055]-[0056] and in the exemplary figs. 8A-B; see in [0072] the initial presentation of the imagery with the first portion (page or slide) of the document in the online meeting; especially note the first row in figs. 9A-B].
Sharp does not explicitly teach:
feeding at least image data comprising an image depicting an environment of the first participant during the conference call as input to a machine learning model;
obtaining one or more outputs of the machine learning model, the one or more outputs comprising an indication of a level of confidence that the one or more pixels of the image data correspond to the first participant;
determining whether the level of confidence satisfies a confidence criterion; or
responsive to determining that the level of confidence satisfies the confidence criterion, generating an image portion composed of the one or more pixels and depicting the first participant.
Nguyen teaches feeding image data comprising an image depicting a participant during a conference call as input to a machine learning model [see e.g. col. 6, lines 36-41; note also col. 5, lines 57-58 indicating a camera capturing an image including a person and their surroundings; note the exemplary video conferencing environment in col. 1, lines 1-34].
Nguyen further teaches obtaining one or more outputs of the machine learning model, the one or more outputs comprising an indication of a level of confidence that the one or more pixels of the image data correspond to the participant [see e.g. col. 13, lines 42-49 clearly describing a confidence level for pixels of the user versus the background and that this confidence map may be generated as a byproduct of the machine learning Segmentation]; determining whether the level of confidence satisfies a confidence criterion; and responsive to determining that the level of confidence satisfies the confidence criterion, generating an image portion composed of the one or more pixels and depicting the participant [see col. 10, lines 9-11 indicating a segmented image; especially note the pixel-level labelling indicated in col. 10, line 22; note in col. 2, lines 33-36 the extraction of pixels representing the user; note from col. 13, lines 42-49 the per-pixel output of the machine learning model; note from col. 4, lines 22-25 indicating filtering pixels based on a confidence level and a confidence threshold, such that only pixels whose confidence level satisfies the criterion of being at or exceeding the threshold are not excluded from the segmented image portion].
It would have been obvious to one of ordinary skill in the art having the teachings of Sharp and Nguyen, before the effective filing date of the claimed invention to utilize the machine-learning-based image segmentation technique taught by Nguyen within the framework taught by Sharp for document sharing with participant overlay by feeding the image data to the machine learning model, obtaining one or more outputs of the machine learning model, the one or more outputs comprising an indication of a level of confidence that the one or more pixels of the image data correspond to the first participant, determining whether the level of confidence satisfies a confidence criterion, and responsive to determining that the level of confidence satisfies the confidence criterion, generating an image portion composed of the one or more pixels and depicting the first participant.
One of ordinary skill in the art would have recognized that applying Nguyen’s technique of machine-learning-based image segmentation to Sharp’s environment would have yielded the predictable results of promoting a more efficient image segmentation, which would enhance the quality of the user experience, as per col. 1, lines 48-52 of Nguyen.
The rationale for the combination would be that a particular known technique was recognized as part of the ordinary capabilities of one skilled in the art. One of ordinary skill in the art would have been capable of applying this known technique to a known invention that was ready for improvement and the results would have been predictable to one of ordinary skill in the art.
See MPEP 2143 I.D.
Regarding independent claims 8 and 14, they are analogously rejected as being unpatentable over Sharp in view of Nguyen and Harman.
For independent claim 8, Sharp also teaches a system [see e.g. the system in fig. 1 comprising multiple computing devices, each of them such as the one shown in fig. 2; see [0003]- [0004]] comprising: a memory device [see memory element(s) 204 in fig. 2]; and a set of processing devices coupled to the memory devices [see processor(s) 202 in fig.2 and described in [0028] and note the interconnection with 204], the set of processing device to perform operations as those of the method of independent claim 1.
For independent claim 14, Sharp also teaches a non-transitory computer-readable storage medium comprising instructions [see e.g. [0018]] for a server [note also the meeting server, e.g. in [0018]] that, when executed by a set of processing devices [see e.g. in fig. 1 the multiple computing devices, each of them such as the one shown in fig. 2], cause the set of processing devices to perform operations as those of the method of independent claim 1.
Regarding claims 3, 10, and 16, the rejection of independent claims 1, 8, and 14 are respectively incorporated. Sharp further teaches that the second content is distinct from the first content [again, note from [0072] the first and second pages or slides of a document being presented, which are distinct from one another].
Regarding claims 4, 11, and 17, the rejection of independent claims 1, 8, and 14 are respectively incorporated. Sharp further teaches that generating the view of the electronic document comprises generating an indication of one or more regions of the electronic document that satisfy an image placement criterion, the one or more regions comprising the at least one region [see e.g. [0050] and note identifying the shape, size/dimensions, and spatial location of overlay window 750 within the slide].
Regarding claims 5 and 12, the rejection of claims 1 and 8 are respectively incorporated.
Nguyen further teaches extracting, from the image data, the set of one or more pixels corresponding to the first participant based on the one or more outputs of the machine learning model [note in col. 2, lines 33-36 the extraction of pixels representing the user; note from col. 13, lines 42-49 the per-pixel output of the machine learning model]. See the rejection of the independent claims for motivations to combine the cited art.
Regarding claim 7, the rejection of independent claim 1 is incorporated.
Sharp further teaches:
receiving an additional request to move the image portion depicting the first participant from the at least one region of the electronic document to an additional region of the electronic document [see e.g. in fig. 9C the movement of the overlay window 750A to the location shown as 750B, as described in [0059]; note also the clicking on and dragging of the window to another location within the slide/document]; and
presenting the image portion depicting the first participant at the additional region of the electronic document [see in fig. 9D the corresponding movement of the presentation window 850A presenting the image depicting the presenter to the location shown as 850B, as also described in [0059]].
Claims 6, 13, 21, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Sharp in view of Nguyen, as applied to claim 1 (for claims 1, 21, and 24) and claim 8 (for claim 13), and further in view of Harman et al., US PGPUB 2003/0050931 A1 (hereinafter as Harman).
Regarding claims 6 and 13, the rejection of independent claims 1 and 8 are respectively incorporated.
The previously combined art does not explicitly teach obtaining, from the second client device, hardware constraint data indicating one or more hardware constraints associated with that device
Harman teaches obtaining, from the receiving client device, hardware constraint data indicating one or more hardware constraints associated with that device [see e.g. [0051] indicating gathering data indicating device parameters; see e.g. the abstract indicating a descriptor defining parameters of a display environment; especially note in [0024] examples of hardware constraints associated with the device such as memory buffer and screen size].
It would have been obvious to one of ordinary skill in the art having the teachings of Sharp and Harman, before the effective filing date of the claimed invention to apply Harman’s approach for dealing with a receiving viewing device having one or more hardware constraints to the second client device in the collaborative environment taught by Sharp in the process of sharing a document with distinct presentable content portions with an image overlay. The motivation for this obvious combination of teachings would be to enable displaying arbitrary content in a way that is compatible with the capabilities of different viewing client devices and to enable adding content (such as the image) that appears on each displayed/displayable page on the viewing device in a template form, which would allow for providing more sophisticated display environments that incorporate practicality and readability of document sharing across different platforms that belong to participants in a collaboration, as suggested by Harman [see e.g. abstract, fig. 5, step 504, [0147], and [0159]].
Regarding claim 21, the rejection of claim 6 is incorporated.
Harman further teaches that the hardware constraint data comprises at least one of an image resolution constraint of the receiving client device or a screen size of the receiving client device [see e.g. [0051] indicating gathering data indicating device parameters; especially note in [0024] examples of hardware constraints associated with the device such as memory buffer and screen size]. See the rejection of claim 6 for motivations to combine the cited art.
Regarding claim 24, the rejection of claim 6 is incorporated.
Sharp further teaches:
that the generated view provided, during the conference call for presentation via the second GUI, comprises the image portion depicting the first participant and the first content of the electronic document [again, note e.g. in fig. 6, steps 640 and 650 indicating the integration of the video content (including the image depicting the first participant as per the portions cited above) within the shared document and providing them for presentation via a remote GUI on a remote client device; again see also the description in [0055]-[0056] and in the exemplary figs. 8A-B; see in [0072] the initial presentation of the imagery with the first portion (page or slide) of the document in the online meeting; especially note the first row in figs. 9A-B];
detecting, during the conference call, that the first participant shifted focus from the first content to the second content of the electronic document [again, see in [0072] the shifting of focus of a presenter from a first page or slide to a second one in a document in the online meeting and an indicative notification of that shifting to the meeting server]; and
updating the generated view to comprise the image portion depicting the first participant and the second content of the electronic document [note e.g. in fig. 6, steps 640 and 650 indicating the integration of the video content (including the image depicting the first participant as per the portions cited above) within the shared document and providing them for presentation via a remote GUI on a remote client device; again see also the description in [0055]-[0056] and in the exemplary figs. 8A-B; again, see in [0072] the presentation of the imagery with the second portion (page or slide) responsive to the shifting of focus of a presenter from the first page or slide to the second one in the document in the online meeting; especially note the second row in figs. 9A-B].
The previously combined art, however, does not explicitly teach that the providing of the generated view comprising the image portion depicting the image portion depicting the first participant and the first content of the electronic document is based on hardware constraint data associated with the second client device.
Harman teaches presenting a document and an image via a GUI for a receiving viewing device, wherein the presentation comprises presenting, with the image, an identified first portion of a certain piece of digital content without presenting an identified second portion of the piece of digital content, and wherein this presentation split is based on hardware constraint data associated with the receiving viewing device [see e.g. [0051] indicating gathering data indicating device parameters; especially note in [0024] examples of hardware constraints associated with the device; see e.g. [0159] describing the display of a company logo (corresponding to an overlaid image) with a displayable page of content (which includes only an identified portion of a certain piece of digital content (corresponding to only a first portion of the document, i.e. the second portion of the document which is in a following displayable page according to Harman’s partitioning will not be initially presented; see [0147] of Harman and fig. 5 for the splitting of document into multiple pages; especially note in.[0024] the splitting of an item across multiple pages (each page having a portion of the item content) as well as steps 504, 506, and 508 of fig. 5 and the description in [0145]-[0148] applied to splitting into two pages of content portions)].
Examiner notes that Harman further teaches updating the GUI for the viewing device to present the identified second portion of the piece of digital content and the image [again see e.g. [0159] and note the display of varying displayable pages of the content (including any of the multiple pages) with the display of the company logo (corresponding to the overlaid image); note that in an updated display, only a second portion of the document according to Harman’s partitioning will be presented; again see [0147] of Harman and fig. 5 for the splitting of document into multiple pages].
Again, it would have been obvious to one of ordinary skill in the art having the teachings of Sharp and Harman, before the effective filing date of the claimed invention to further apply Harman’s approach for dealing with a receiving viewing device with one or more hardware constraints to the second client device in the collaborative environment taught by Sharp in the process of sharing a document with distinct presentable content portions with an image overlay. The motivation for this obvious combination of teachings would be to enable displaying arbitrary content in a way that is compatible with the capabilities of different viewing client devices and to enable adding content (such as the image) that appears on each displayed/displayable page on the viewing device in a template form, which would allow for providing more sophisticated display environments that incorporate practicality and readability of document sharing across different platforms that belong to participants in a collaboration, as suggested by Harman [see e.g. abstract, fig. 5, step 504, [0147], and [0159]].
Claim 25 is rejected under 35 U.S.C. 103 as being unpatentable over Sharp in view of Nguyen, as applied to claim 1 above, and further in view of Chau, US Patent No. 10,445,772 B1 (hereinafter as Chau).
Regarding claim 25, the rejection of claim 1 is incorporated. The previously combined art does not explicitly teach that determining the at least one region of the electronic document for inclusion of the image portion based on a size of the one or more pixels.
Chau teaches determining at least one region of an electronic document for inclusion of an image portion based on a size of the image portion [see e.g. col. 5, lines 49-59 indicating determining a location (at the very top or at the very bottom of the electronic page (image)) that satisfies a placement criteria position to place an image overlay (advertisement) wherein the placement position is based on a size of the image overlap].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Chau before the effective filing date of the claimed invention to apply Chau’s teaching regarding determining at least one region of an electronic document for inclusion of an image portion based on a size of the image portion to the electronic document taught by Fogarty and to the segmented image in the form of extracted pixel set) returned by the machine learning model taught by Nguyen, by specifying determining the at least one region of the electronic document for inclusion of the image portion based on a size of the one or more pixels, as per the teachings of Chau with respect to the modification of an electronic page based on placement of additional advertisement. The motivation for this obvious combination of teachings would be to enable the addition of content in a way that does not interfere with the user’s experience should it obstruct objects of interest in the document, as suggested by Chau [see e.g. col. 1, lines 31-34 and col. 5, lines 60-67].
Claim 26 is rejected under 35 U.S.C. 103 as being unpatentable over Sharp in view of Nguyen, as applied to claim 1 above, and further in view of KANSARA et al., US PGPUB 2018/0096502 A1 (hereinafter as Kansara).
Regarding claim 26, the rejection of claim 1 is incorporated. The previously combined art does not explicitly teach determining the at least one region of the electronic document for inclusion of the image portion based on a difference between a color of the one or more pixels and a color of one or more of the first content, the second content, or a background of the electronic document.
Kansara teaches determining at least one region of an electronic document for inclusion of an image portion based on a difference between a color of the one or more pixels of the image portion and a color of one or more portions/regions/subsets of the electronic document [see fig. 11, steps 1106-1108 and note from [0077] and [0078] the selection of a region from the allowed regions having a numeric value that is based on the color contrast analysis (between pixels of the image to be inserted and those of the current region in the content document); note in the last 5 lines of [0078] weighting the color contrast analysis more heavily].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and KANSARA before the effective filing date of the claimed invention to modify the electronic document region selection in Sharp by explicitly specifying determining the at least one region of the electronic document for inclusion of the image portion based on a difference between a color of the one or more pixels of the image portion and a color of one or more of the first content, the second content, or a background of the electronic document, as per the teachings of KANSARA. The motivation for this obvious combination of teachings would be to enable clear visibility and good contrast with surrounding portions of the base content, as suggested by KANSARA [see e.g. [0003] and [0078]].
Claim 27 is rejected under 35 U.S.C. 103 as being unpatentable over Sharp in view of Nguyen, as applied to claim 1 above, and further in view of Agarwal et al., US PGPUB 2016/0148428 Al (hereinafter as Agarwal).
Regarding claim 27, the rejection of claim 1 is incorporated. The previously combined art does not explicitly teach that the image portion included in the generated view has a different color temperature from a color temperature of the one or more pixels of the image data based on a color temperature associated with the electronic document.
Agarwal teaches an image portion included in a generated view that has a different color temperature from a color temperature of the one or more pixels of the corresponding image data (prior to including it the generated view) based on a color temperature associated with a background electronic document [see e.g. [0047]-[0048]; [0050]; and [0082]; note adjusting the color temperature of pixels of a cutout object (that is overlaid on another captured image) to match the color temperature of the captured image].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Agarwal before the effective filing date of the claimed invention to modify Sharp’s image portion included in the generated view by explicitly specifying that the image portion included in the generated view has a different color temperature from a color temperature of the one or more pixels of the image data based on a color temperature associated with the electronic document, as per the teachings of Agarwal. The motivation for this obvious combination of teachings would be to promote an enhanced visual appearance of the final output by creating an appearance as if the overlaid image portion is part of the overall scene, as suggested by Agarwal [see e.g. [0050]].
Claim 28 is rejected under 35 U.S.C. 103 as being unpatentable over Sharp in view of Nguyen, as applied to claim 1 above, and further in view of Fogarty et al., US PGPUB 2016/0260236 Al (hereinafter as Fogarty).
Regarding claim 28, the rejection of claim 1 is incorporated. The previously combined art does not explicitly teach that the image portion included in the generated view has a different transparency than a transparency of the one or more pixels of the image data such that content items at the at least one region of the electronic document are detectable when the image portion is presented at the at least one region.
Fogarty teaches an image portion included in a generated view (including an electronic document and an overlaid image) wherein the overlaid image has a different transparency than a transparency of the image data (before being overlaid) such that content items at the at least one region of the electronic document are detectable when the overlaid image portion is presented at the at least one region [see e.g. [0024] indicating increasing the transparency of an image of a presenter when overlaid on a presentation including other content; see also fig. 2, 210 and 212 and [0013]].
It would have been obvious to one of ordinary skill in the art having the teachings of the previously combined art and Fogarty, before the effective filing date of the claimed invention to modify Sharp’s image portion included in the generated view by explicitly specifying having a different transparency than a transparency of the one or more pixels of the extracted image data such that content items at the at least one region of the electronic document are detectable when the image portion is presented at the at least one region, as per the teachings of Fogarty. The motivation for this obvious combination of teachings would be to reduce obscuring effects of the presenter over the displayed presentation, thus maintaining presentation quality for remote participants while also enabling a presenter’s view which would enhance the presentation experience, as suggested by Fogarty [again see [0019] and [0024]].
Response to Arguments
Applicant's prior art arguments with respect to amended independent claim 1 have been fully considered but they are not persuasive.
Applicant argues that “Nguyen merely teaches filtering applied to an already-segmented frame to "exclude pixels whose confidence level falls below a confidence threshold." However, Nguyen is silent regarding "obtaining one or more outputs of the machine learning model, the one or more outputs comprising an indication of a level of confidence that the one or more pixels of the image data correspond to the first participant,' "determining whether the level of confidence satisfies a confidence criterion," and "responsive to determining that the level of confidence satisfies the confidence criterion, generating a view of the electronic document comprising an image portion composed of the one or more pixels and depicting the first participant in at least one region of the electronic document with at least one of the first content or the second content."” [See p. 11 of the Response].
First, Examiner respectfully notes that the limitations, as initiated above are not supported by Applicant’s original disclosure, and therefore are rejected under 35 U.S.C. 112(a) as failing to comply with the written description requirement. Applicant is respectfully referred to that rejection above as well as the claim interpretation that is supported by Applicant’s disclosure.
Next, Examiner respectfully notes that the primary reference, Sharp, clearly and explicitly teaches obtaining an image portion (extracted from the image data) depicting the first participant and generating a view of the electronic document comprising the image portion depicting the first participant in at least one region of the electronic document with at least one of the first content or the second content and providing, during the conference call and for presentation via the second GUI, the generated view of the electronic document [see e.g. [0055]-[0056] and note e.g. in fig. 6, steps 640 and 650 indicating the integration of the video content (including the image depicting the first participant as per the portions cited above) within the shared document and providing them for presentation via a remote GUI on a remote client device].
The missing elements in the above-indicated limitations from Sharp’s teachings is the use of a machine learning model including the use of a confidence criteria in conjunction with output confidence levels for the pixels to generate an image portion composed of one or more pixels that depicts the participant.
Nguyen clearly teaches the use of a machine learning model with as input that comprises an image depicting a participant during a conference call [see e.g. col. 6, lines 36-41; note also col. 5, lines 57-58 indicating a camera capturing an image including a person and their surroundings; note the exemplary video conferencing environment in col. 1, lines 1-34] and one or more outputs comprising an indication of a level of confidence that the one or more pixels of the image data correspond to the participant [see e.g. col. 13, lines 42-49 clearly describing a confidence level for pixels of the user versus the background and that this confidence map may be generated as a byproduct of the machine learning Segmentation].
Nguyen further teaches determining whether the level of confidence satisfies a confidence criterion; and responsive to determining that the level of confidence satisfies the confidence criterion, generating an image portion composed of the one or more pixels and depicting the participant [see col. 10, lines 9-11 indicating a segmented image; especially note the pixel-level labelling indicated in col. 10, line 22; note in col. 2, lines 33-36 the extraction of pixels representing the user; note from col. 13, lines 42-49 the per-pixel output of the machine learning model; note from col. 4, lines 22-25 indicating filtering pixels based on a confidence level and a confidence threshold, such that only pixels whose confidence level satisfies the criterion of being at or exceeding the threshold are not excluded from the final segmented image portion].
The rejection then clearly indicates that it would have been obvious to one of ordinary skill in the art having the teachings of Sharp and Nguyen, before the effective filing date of the claimed invention to utilize the machine-learning-based image segmentation technique taught by Nguyen within the framework taught by Sharp for document sharing with participant overlay by feeding the image data to the machine learning model, obtaining one or more outputs of the machine learning model, the one or more outputs comprising an indication of a level of confidence that the one or more pixels of the image data correspond to the first participant, determining whether the level of confidence satisfies a confidence criterion, and responsive to determining that the level of confidence satisfies the confidence criterion, generating an image portion composed of the one or more pixels and depicting the first participant thus reaching the claim limitations, as amended.
Examiner reminds Applicant that one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Thus, Examiner respectfully asserts that the combination of the cited art sufficiently teaches each and every limitations recited in amended independent claim 1. Analogous rationale holds for amended independent claims 8 and 14. For a more detailed analysis, please refer to the full rejections above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Examiner notes from the cited art:
Zhao et al., US PG PUB 20200304713 A1, which teaches a pixel-based segmentation model [see e.g. [0056]-[0057] and fig. 5].
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARIA S AYAD whose telephone number is (571)272-2743. The examiner can normally be reached Monday-Friday, 7:30 am - 4:30 pm. Alt, Friday, EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Adam Queler can be reached at (571) 272-4140. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARIA S AYAD/Primary Examiner, Art Unit 2172