DETAILED ACTION
Information Disclosure Statement
1. The information disclosure statements submitted are being considered by the examiner.
Double Patenting
2. The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
3. Claims 1-3, 5, 9-13 and 15-18 are rejected under the judicially created doctrine of obviousness-type double patenting as being unpatentable over claims 1-3, 7, 8, 10 and 15 of U.S. Patent No. 12,231,803 in view of Nicholson et al, U.S. Patent Application Publication No. 2023/0306610 (hereinafter Nicholson).
Regarding claim 1 of the instant application, claim 1 of U.S. Patent No. 12,231,803 discloses a method, comprising:
identifying, using software-based image processing applied to camera-generated visual data from a client device, foreground imagery representing a participant and background imagery representing content of the camera-generated visual data other than the foreground imagery;
determining an identity of an extraneous item comprising a portion of the background imagery;
determining based on the identity of the extraneous item, to replace the extraneous item in the camera-generated visual data;
predicting, using replacement imagery prediction software, replacement imagery to replace the extraneous item; and
generating a composite image comprising the foreground imagery of the camera-generated visual data and the background imagery with the extraneous item replaced by the replacement imagery.
Still on the issue of claim 1 of the instant application, claim 1 of U.S. Patent No. 12,231,803 does not teach the determining steps are performed by a neural network. All the same, Nicholson discloses the determining steps are performed by a neural network (from paragraph 0040, see the one or more processors 112 may include an artificial neural network 122). Therefore, it would have been obvious to one of ordinary skill in the art to modify claim 1 of U.S. Patent No. 12,231,803 wherein the determining steps are performed by a neural network as taught by Nicholson. This modification would have improved the system’s efficiency by providing for learning from feedback as suggested by Nicholson.
Regarding claim 2 of the instant application, claim 2 of U.S. Patent No. 12,231,803 discloses the camera-generated visual data comprises at least one frame in a video generated by a camera at the client device.
Regarding claim 3 of the instant application, claim 1 of U.S. Patent No. 12,231,803 discloses receiving, during a video conference to which the client device is connected, the camera-generated visual data for output to at least one remote device participating in the video conference; and transmitting the composite image to the at least one remote device during the video conference.
Regarding claim 5 of the instant application, the combination of claim 1 of U.S. Patent No. 12,231,803 and Nicholson discloses wherein determining to remote the extraneous item from the camera-generated visual data comprises prompting, before generating the composite image, a user of the client device to approve or deny replacement of the extraneous item and receiving, from the client device, approval to replace the extraneous item (from paragraph 0045 of Nicholson, see the user may select, via the input device 108, one or more of the graphic indicators 402 and/or objects 310 to set a status for the selected objects 310 as “keep” or “modify”. If the user desires to conceal a first object 310, then the user may control the input device 108 to provide a user input selection of the graphic indicator 402 associated with the first object 310. For example, the user may “click” on the graphic indicator 402 associated with the first object 310 using the input device 108. In response to receiving the user input selection, the controller 102 may generate a graphical dialog box that is displayed on the display device 110. The graphical dialog box may prompt the user to select whether to keep the first object 310 in the background environment 304 or to modify the first object 310 to conceal the first object 310 within the video feed that is remotely communicated. The controller 102 modifies the objects 310 that are identified for modification by changing the appearance of those objects 310 in output image data, relative to the appearance of those objects 310 in the input image data. The controller 102 may not modify the appearance of the objects 310 that are not identified for modification. After setting the status of the first object 310, the user may select another graphic indicator 402 to set the status of another one of the objects 310, if desired. In an embodiment, the default setting for the objects 310 may be to keep as is, unless the user changes the status).
Regarding claim 9 of the instant application, the combination of claim 1 of U.S. Patent No. 12,231,803 and Nicholson discloses predicting the replacement imagery comprises using a rule-based technique to extend a pattern (from paragraph 0052 of Nicholson, see For example, the controller 102 may extrapolate the content of the background environment 304 surrounding the space, such that the content continues into the space and fills the space. In an embodiment, if the object 310 to be extracted is surrounds by pixels having approximately the same color, then the controller 102 may simply fill in the space by adding pixels of that same color).
Regarding claim 10 of the instant application, the combination of claim 7 of U.S. Patent No. 12,231,803 and Nicholson discloses determining to replace the extraneous item in the camera-generated visual data comprises determining, by the neural network, the identity of the extraneous item corresponds to extraneous items replaced by other users.
Regarding claim 11 of the instant application, claim 3 of U.S. Patent No. 12,231,803 discloses detecting the participant moves in front of the extraneous item, wherein the composite image depicts the participant obscuring a co-located part of the replacement imagery.
Regarding claim 12 of the instant application, claim 8 of U.S. Patent No. 12,231,803 discloses a non-transitory computer readable medium storing instructions operable to cause one or more processors to perform operations comprising:
identifying, using software-based image processing applied to camera-generated visual data from a client device, foreground imagery representing a participant and background imagery representing content of the camera-generated visual data other than the foreground imagery;
determining an identity of an extraneous item comprising a portion of the background imagery;
determining based on the identity of the extraneous item, to replace the extraneous item in the camera-generated visual data;
predicting, using replacement imagery prediction software, replacement imagery to replace the extraneous item; and
generating a composite image comprising the foreground imagery of the camera-generated visual data and the background imagery with the extraneous item replaced by the replacement imagery.
Still on the issue of claim 12 of the instant application, claim 8 of U.S. Patent No. 12,231,803 does not teach the determining steps are performed by a neural network. All the same, Nicholson discloses the determining steps are performed by a neural network (from paragraph 0040, see the one or more processors 112 may include an artificial neural network 122). Therefore, it would have been obvious to one of ordinary skill in the art to modify claim 8 of U.S. Patent No. 12,231,803 wherein the determining steps are performed by a neural network as taught by Nicholson. This modification would have improved the system’s efficiency by providing for learning from feedback as suggested by Nicholson.
Regarding claim 13 of the instant application, claim 8 of U.S. Patent No. 12,231,803 discloses receiving, during a video conference to which the client device is connected, the camera-generated visual data for output to at least one remote device participating in the video conference; and transmitting the composite image to the at least one remote device during the video conference.
As per claim 15 of the instant application, although this claim is not identical to claims 1 and 7 of U.S. Patent No. 12,231,803, they are not patentably distinct from each other claim 15 of the instant application is directed towards a non-transitory computer readable medium while claims 1 and 7 of U.S. Patent No. 12,231,803 are directed towards a method. Indeed, the non-transitory computer-readable medium of the instant application can be used to practice the method of the patent. Hence, claim 15 of the instant application is an obvious variant of claims 1 and 7 of U.S. Patent No. 12,231,803.
Regarding claim 16 of the instant application, claim 10 of U.S. Patent No. 12,231,803 discloses detecting the participant moves in front of the extraneous item, wherein the composite image depicts the participant obscuring a co-located part of the replacement imagery.
Regarding claim 17 of the instant application, claim 15 of U.S. Patent No. 12,231,803 discloses an apparatus, comprising:
a memory; and
a processor configured to execute instructions stored in the memory to:
identify, using software-based image processing applied to camera-generated visual data from a client device, foreground imagery representing a participant and background imagery representing content of the camera-generated visual data other than the foreground imagery;
determine an identity of an extraneous item comprising a portion of the background imagery;
determine based on the identity of the extraneous item, to replace the extraneous item in the camera-generated visual data;
predict, using replacement imagery prediction software, replacement imagery to replace the extraneous item; and
generate a composite image comprising the foreground imagery of the camera-generated visual data and the background imagery with the extraneous item replaced by the replacement imagery.
Still on the issue of claim 17 of the instant application, claim 15 of U.S. Patent No. 12,231,803 does not teach the determining steps are performed by a neural network. All the same, Nicholson discloses the determining steps are performed by a neural network (from paragraph 0040, see the one or more processors 112 may include an artificial neural network 122). Therefore, it would have been obvious to one of ordinary skill in the art to modify claim 15 of U.S. Patent No. 12,231,803 wherein the determining steps are performed by a neural network as taught by Nicholson. This modification would have improved the system’s efficiency by providing for learning from feedback as suggested by Nicholson.
Claim 18 is rejected for the same reasons as claim 5.
Claim Rejections - 35 USC § 102
4. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
5. Claims 1-3, 5, 9, 10, 12, 13, 15 and 17 are rejected under 35 U.S.C. § 102(a)(2) as being anticipated by Nicholson.
Regarding claim 1, Nicholson discloses a method, comprising:
identifying, using software-based image processing applied to camera-generated visual data from a client device, foreground imagery representing a participant and background imagery representing content of the camera-generated visual data other than the foreground imagery (from paragraph 0039, see In an embodiment, the controller 102 analyzes and segments the received input image data. For example, the controller 102 may perform a first segmentation to differentiate the foreground environment 302 from the background environment 304);
determining, by a neural network, an identity of an extraneous item comprising a portion of the background imagery (from paragraph 0039, see The controller 102 may perform a second segmentation to locate objects 310 in the background environment 304);
determining, by the neural network based on the identity of the extraneous item, to replace the extraneous item in the camera-generated visual data (from paragraph 0043, see After the segmentation of the input image data, the controller 102 may identify a set of one or more of the objects 310 in the background environment 304 to modify. For example, some of the objects 310 may be deemed by the user as too personal or private, or simply not appropriate for the tenor of the video conference call. The set of objects 310 may be identified in order to conceal those objects 310 from view by persons that view the video stream showing the room 308);
predicting, using replacement imagery prediction software, replacement imagery to replace the extraneous item (from paragraph 0052, see In an embodiment, the controller 102 may generate new image data to fill the space in the background environment 304 that corresponds to the location of the object 310 that is extracted. For example, the controller 102 may extrapolate the content of the background environment 304 surrounding the space, such that the content continues into the space and fills the space); and
generating a composite image comprising the foreground imagery of the camera-generated visual data and the background imagery with the extraneous item replaced by the replacement imagery (from paragraph 0049, FIG. 5 illustrates an output image 500 that is based on image data generated by the controller 102 of the image alteration system 100 according to an embodiment. The image data generated by the controller 102 is referred to herein as output image data. The output image 500 depicts a modified version 502 of the background environment 304. The output image 500 (e.g., output image data) represents the background portion of the image data that is remotely transmitted during the video streaming event. As such, a participant in a video conference using one of the remote computer devices 204 would view the output image 500 shown in FIG. 5, rather than the input image 300 that is shown in FIG. 3 and the annotated image 400 shown in FIG. 4. The controller 102 generates the output image data by changing the appearance of the objects 310 that are identified for modification. The appearance change may be based on the mode of concealment selected for each of the objects 310 identified for modification).
Regarding claim 2, Nicholson discloses the camera-generated visual data comprises at least one frame in a video generated by a camera at the client device (from paragraph 0021, see The input image data may be one or more image frames of a video stream that is generated by a camera).
Regarding claim 3, Nicholson discloses receiving, during a video conference to which the client device is connected, the camera-generated visual data for output to at least one remote device participating in the video conference and transmitting the composite image to the at least one remote device during the video conference (from paragraph 0049, see As such, a participant in a video conference using one of the remote computer devices 204 would view the output image 500 shown in FIG. 5, rather than the input image 300 that is shown in FIG. 3 and the annotated image 400 shown in FIG. 4).
Regarding claim 5, Nicholson discloses wherein to determine to remove the extraneous item from the camera-generated visual data comprises to: prompt, before generating the composite image, a user of the client device to approve or deny replacement of the extraneous item; and receive, from the client device, approval to replace the extraneous item (from paragraph 0045, see the user may select, via the input device 108, one or more of the graphic indicators 402 and/or objects 310 to set a status for the selected objects 310 as “keep” or “modify”. If the user desires to conceal a first object 310, then the user may control the input device 108 to provide a user input selection of the graphic indicator 402 associated with the first object 310. For example, the user may “click” on the graphic indicator 402 associated with the first object 310 using the input device 108. In response to receiving the user input selection, the controller 102 may generate a graphical dialog box that is displayed on the display device 110. The graphical dialog box may prompt the user to select whether to keep the first object 310 in the background environment 304 or to modify the first object 310 to conceal the first object 310 within the video feed that is remotely communicated. The controller 102 modifies the objects 310 that are identified for modification by changing the appearance of those objects 310 in output image data, relative to the appearance of those objects 310 in the input image data. The controller 102 may not modify the appearance of the objects 310 that are not identified for modification. After setting the status of the first object 310, the user may select another graphic indicator 402 to set the status of another one of the objects 310, if desired. In an embodiment, the default setting for the objects 310 may be to keep as is, unless the user changes the status).
Regarding claim 9, Nicholson discloses predicting the replacement imagery comprises using a rule-based technique to extend a pattern (from paragraph 0052, see For example, the controller 102 may extrapolate the content of the background environment 304 surrounding the space, such that the content continues into the space and fills the space. In an embodiment, if the object 310 to be extracted is surrounds by pixels having approximately the same color, then the controller 102 may simply fill in the space by adding pixels of that same color).
Regarding claim 10, Nicholson discloses wherein determining to replace the extraneous item in the camera-generated visual data comprises determining, by the neural network, the identity of the extraneous item corresponds to extraneous items replaced by other users (from paragraph 0025, see References herein to “machine learning” and “artificial intelligence” refer to algorithms that learn from various automatic or manual feedback, such as observations and/or data. The artificial intelligence algorithms may be adjusted over multiple iterations based on the observations and/or data. For example, the artificial intelligence algorithms may be adjusted by supervised learning, unsupervised learning, and/or reinforcement learning (e.g., customer feedback). Non-limiting examples of artificial intelligence algorithms include decision trees, K-means, deep learning, artificial neural networks, and/or the like).
Regarding claim 12, Nicholson discloses a non-transitory computer readable medium (from paragraph 0067, see readable medium(s)) storing instructions operable to cause one or more processors (from paragraph 0069, see processor) to perform operations comprising:
identifying, using software-based image processing applied to camera-generated visual data from a client device, foreground imagery representing a participant and background imagery representing content of the camera-generated visual data other than the foreground imagery (from paragraph 0039, see In an embodiment, the controller 102 analyzes and segments the received input image data. For example, the controller 102 may perform a first segmentation to differentiate the foreground environment 302 from the background environment 304);
determining, by a neural network, an identity of an extraneous item comprising a portion of the background imagery (from paragraph 0039, see The controller 102 may perform a second segmentation to locate objects 310 in the background environment 304);
determining, by the neural network based on the identity of the extraneous item, to replace the extraneous item in the camera-generated visual data (from paragraph 0043, see After the segmentation of the input image data, the controller 102 may identify a set of one or more of the objects 310 in the background environment 304 to modify. For example, some of the objects 310 may be deemed by the user as too personal or private, or simply not appropriate for the tenor of the video conference call. The set of objects 310 may be identified in order to conceal those objects 310 from view by persons that view the video stream showing the room 308).
predicting, using replacement imagery prediction software, replacement imagery to replace the extraneous item (from paragraph 0052, see In an embodiment, the controller 102 may generate new image data to fill the space in the background environment 304 that corresponds to the location of the object 310 that is extracted. For example, the controller 102 may extrapolate the content of the background environment 304 surrounding the space, such that the content continues into the space and fills the space);
generating a composite image comprising the foreground imagery of the camera-generated visual data and the background imagery with the extraneous item replaced by the replacement imagery (from paragraph 0049, FIG. 5 illustrates an output image 500 that is based on image data generated by the controller 102 of the image alteration system 100 according to an embodiment. The image data generated by the controller 102 is referred to herein as output image data. The output image 500 depicts a modified version 502 of the background environment 304. The output image 500 (e.g., output image data) represents the background portion of the image data that is remotely transmitted during the video streaming event. As such, a participant in a video conference using one of the remote computer devices 204 would view the output image 500 shown in FIG. 5, rather than the input image 300 that is shown in FIG. 3 and the annotated image 400 shown in FIG. 4. The controller 102 generates the output image data by changing the appearance of the objects 310 that are identified for modification. The appearance change may be based on the mode of concealment selected for each of the objects 310 identified for modification).
Claim 13 is rejected for the same reasons as claim 3.
Claim 15 is rejected for the same reasons as claim 9.
Regarding claim 17, Nicholson discloses an apparatus (from paragraph 0067, see apparatus), comprising:
a memory (from paragraph 0067, see memory); and
a processor (from paragraph 0069, see processor) configured to execute instructions stored in the memory to:
identify, using software-based image processing applied to camera-generated visual data from a client device, foreground imagery representing a participant and background imagery representing content of the camera-generated visual data other than the foreground imagery (from paragraph 0039, see In an embodiment, the controller 102 analyzes and segments the received input image data. For example, the controller 102 may perform a first segmentation to differentiate the foreground environment 302 from the background environment 304);
determine, by a neural network, an identity of an extraneous item comprising a portion of the background imagery (from paragraph 0039, see The controller 102 may perform a second segmentation to locate objects 310 in the background environment 304);
determine, by the neural network based on the identity of the extraneous item, to replace the extraneous item in the camera-generated visual data (from paragraph 0043, see After the segmentation of the input image data, the controller 102 may identify a set of one or more of the objects 310 in the background environment 304 to modify. For example, some of the objects 310 may be deemed by the user as too personal or private, or simply not appropriate for the tenor of the video conference call. The set of objects 310 may be identified in order to conceal those objects 310 from view by persons that view the video stream showing the room 308);
predict, using replacement imagery prediction software, replacement imagery to replace the extraneous item (from paragraph 0052, see In an embodiment, the controller 102 may generate new image data to fill the space in the background environment 304 that corresponds to the location of the object 310 that is extracted. For example, the controller 102 may extrapolate the content of the background environment 304 surrounding the space, such that the content continues into the space and fills the space).
generate a composite image comprising the foreground imagery of the camera-generated visual data and the background imagery with the extraneous item replaced by the replacement imagery (from paragraph 0049, FIG. 5 illustrates an output image 500 that is based on image data generated by the controller 102 of the image alteration system 100 according to an embodiment. The image data generated by the controller 102 is referred to herein as output image data. The output image 500 depicts a modified version 502 of the background environment 304. The output image 500 (e.g., output image data) represents the background portion of the image data that is remotely transmitted during the video streaming event. As such, a participant in a video conference using one of the remote computer devices 204 would view the output image 500 shown in FIG. 5, rather than the input image 300 that is shown in FIG. 3 and the annotated image 400 shown in FIG. 4. The controller 102 generates the output image data by changing the appearance of the objects 310 that are identified for modification. The appearance change may be based on the mode of concealment selected for each of the objects 310 identified for modification).
Claim 18 is rejected for the same reasons as claim 5.
Claim Rejections - 35 USC § 103
6. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claims 11 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Nicholson et al, U.S. Patent Application Publication No. 2023/0306610 (hereinafter Nicholson) in view of Chu et al, U.S. Patent No. 11,145,334 (hereinafter Chu).
Regarding claim 11, Nicholson does not teach detecting the participant moves in front of the extraneous item, wherein the composite image depicts the participant obscuring a co-located part of the replacement imagery. All the same, Chu discloses detecting the participant moves in front of the item, wherein the composite image depicts the participant obscuring a co-located part of the imagery (from abstract, see At least a portion of the information in the one or more frames is determined to be obscured. In response to this determination, information from audio data and the unobscured portion of the one or more frames is extracted. Using this extracted information, a set of replacement data is generated, according to the information type, for the obscured information in the one or more frames. The set of replacement data and the unobscured portion of the one or more frames is merged into one or more composite frames. These composite frames are displayed to a user). Therefore, it would have been obvious to one of ordinary skill in the art to modify Nicholson with detecting the participant moves in front of the item, wherein the composite image depicts the participant obscuring a co-located part of the imagery as taught by Chu. This modification would have enhanced communication and collaboration by providing an accurate and complete presentation of information as suggested by Chu.
Claim 16 is rejected for the same reasons as claim 11.
Allowable Subject Matter
8. Claims 4, 6-8, 14, 19 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
9. Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLISA ANWAH whose telephone number is 571-272-7533. The examiner can normally be reached Monday to Friday from 8.30 AM to 6 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carolyn Edwards can be reached on 571-270-7136. The fax phone numbers for the organization where this application or proceeding is assigned are 571-273-8300 for regular communications and 571-273-8300 for After Final communications.
Any inquiry of a general nature or relating to the status of this application or proceeding should be directed to the receptionist whose telephone number is 571-272-2600.
Olisa Anwah
Patent Examiner
August 13, 2026
/OLISA ANWAH/Primary Examiner, Art Unit 2692
Prosecution Insights