DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 25 March 2026 has been entered.
Applicant’s arguments with respect to claim(s) 1, 9, and 15 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 102
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1, 9, and 15 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Benchemsi et al. (US 2023/0412785 A1).
Regarding Claims 1, 9, and 15, Benchemsi discloses an apparatus using a CRM to perform a method comprising: acquiring a first video stream from a first camera of a video device [Benchemsi: ¶ [0033] FIG. 8 is a schematic that illustrates operation of the positioner module 414 to create a composite image 802. The positioner module 414 receives the foreground image 502]; detecting a foreground object in the first video stream [Benchemsi: ¶ [0028]: The foreground extractor module 408 extracts the foreground object 404 from the image 402 to create a foreground image 502. The foreground object 404 can be extracted from the image 108 by the foreground extractor module 408 through use of boundary detection technologies that detect a boundary 504 of the foreground object 404 (e.g., the face of a user) so that the foreground image 502 can be created from the pixels of the image 402 found within the boundary 504]; generating a foreground mask video stream based on the foreground object detected in the first video stream [Benchemsi: ¶ [0030] With reference to FIG. 6, a schematic that illustrates operation of the background constructor module 410 is depicted. The background constructor module 410 operates to populate the void 506 in the image 402 left by the extraction of the foreground object 502 by generating a patch image 602 that is the same size and shape as the void 506. The background constructor module 410, upon populating the void 506 with the patch image 602, constructs a background image 604, where the background image 604 includes: 1) a portion of the background object 406 that was not occluded by the foreground object 404 in the image 402; and 2) the patch image 602]; acquiring a second video stream from a second camera of the video device [Benchemsi: ¶ [0038] When a videoconference is in progress, the video frames captured by the cameras 912 and 922 are continuously processed by the videoconferencing applications 918 and 928. For example, the video frames captured by the camera 912 are processed by the video conferencing application 918 to create updated first and second images that are used to generate composite images. The video frames captured by the camera 922 are processed by the video conferencing application 928 to update the location of the user 908 relative to the display 930 that can be sent to the composite image generator system 206 of the first client computing device 902], the second camera being adjusted to be intentionally out of focus, wherein the second video stream is optically blurred [Benchemsi: ¶ [0032] The blurrer module 412 can optionally blur the background image 604… as a consequence of the depth of field of the lens of the camera (i.e., the so-called “bokeh” effect) used to generate the image 402. The blurrer module 412 receives the background image 604 and blurs the background image 604 to smooth the transition between the patch image 602 and the remainder of the background image 604]; determining generating a plurality of optically blurred background image frames from the second video stream over time, wherein the plurality of optically blurred background image frames include different views of a background as the foreground object moves around [Benchemsi: ¶ [0033] FIG. 8 is a schematic that illustrates operation of the positioner module 414 to create a composite image 802. The positioner module 414 receives the foreground image 502, the background image 604, and the location data and generates the composite image 802. As can be seen in FIG. 8, the positioner module 414 has positioned the foreground image 502 relative to the background image 604 such that a portion of the patch image 602 is exposed in the composite image 802. For instance, the user 104 may be positioned towards the right-hand side of the display 100, and accordingly the positioner module 414 exposes more of the background to the right-hand side of the foreground object 404. The positioner module 414, for example, can position the foreground image 502 to be proximate to a center of the display 100 and shift the background image to the right (from the perspective of the user 104), thereby exposing a portion of the patch image 602. The amount of relative movement of the foreground image 502 and the background image 604 is inversely related to the estimated depth between the foreground object 404 and the background object 406; e.g., the greater the depth, the less relative movement between the foreground image 502 and the background image 604 as position of the eyes of the user 104 changes. It should also be noted that the positioning module 414 can operate on individual pixels based on a depth map received with the image 402 so that pixels move differently depending on their relative depth in the scene. Moreover, the composite image generator system 206 can generate several background images, one for each depth plane. The composite image generator system 206 can utilize any suitable technologies in connection with constructing different background images, including boundary detection, use of an infrared sensor to acquire depth data, etc.]; combining the plurality of optically blurred background image frames to form a live artificial background image [Benchemsi: ¶ [0031] Alternatively, a static or dynamic background image (optionally together with a depth map)—such as, for example, an image or video of an artificial background generated by a video conferencing application—can be provided along with the received image 402, and the background constructor module 410 can populate the void 406 with at least a portion of the background image 604. FIG. 7 is a schematic that depicts the background image 604 output by the background constructor module 410]; and combining the foreground mask video stream and the live artificial background image to generate an output video stream that includes the foreground object against an optically blurred background that comprises the live artificial background image [Benchemsi: ¶ [0033]].
Claim Rejections - 35 USC § 103
Claim(s) 2 and 16 is/are rejected under 35 U.S.C. 103 as being unpatentable over Benchemsi as applied to claims 1 and 15 above, and further in view of Kim et al. (US 2020/0051265 A1).
Regarding Claims 2 and 16, Benchemsi discloses all the limitations of Claims 1, 9, and 15, respectively, and is analyzed as previously discussed with respect to those claims.
Benchemsi may not explicitly disclose wherein the second camera is a wide-angle camera.
However, Kim discloses wherein the second camera is a wide-angle camera [Kim: ¶ [0070]; and ¶ [0100]].
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the blur effect of Kim with the blur processing of Benchemsi in order to provide various methods of blur.
Claim(s) 3-7, 10-13, and 17-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Benchemsi as applied to claims 1 and 15 above, and further in view of Hong et al. (US 2013/0242057 A1).
Regarding Claims 3, 10, and 17, Benchemsi discloses all the limitations of Claims 1, 9, and 15, respectively, and is analyzed as previously discussed with respect to those claims.
Benchemsi may not explicitly disclose further comprising determining, from the first video stream, a position of the foreground object and adjusting a focus of the first camera to the position of the foreground object.
However, Hong discloses further comprising determining, from the first video stream, a position of the foreground object and adjusting a focus of the first camera to the position of the foreground object [Hong: ¶ [0114]: At 602, the electronic device 201 focuses the first camera 253a and the second camera 253b at different focus distances. For example, the electronic device 201 may configure the cameras 253a, 253b to operate at different focus distances to capture images. That is, the first camera 253a has a different focus distance than the second camera 253b. In at least some example embodiments, the electronic device 201 may control actuators associated with the cameras to move lenses in each of the cameras 253a, 253b relative to image sensors in the cameras 253a, 253b. Varying the distances between the lenses and the image sensor changes the focus distance of the cameras 253a, 253b when capturing images. Moving the lenses further away from the image sensor, increases the focus distance of the cameras 253a, 253b, while moving the lenses closer to the image sensor decreases the focus distance of the cameras 253a, 253b].
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the focus adjustment of Hong with the process of Benchemsi in order to provide improved image quality.
Regarding Claims 4 and 11, Benchemsi discloses all the limitations of Claims 1, 9, and 15, respectively, and is analyzed as previously discussed with respect to those claims.
Benchemsi may not explicitly disclose further comprising adjusting a focus of the second camera based on a position of the foreground object.
However, Hong discloses further comprising adjusting a focus of the second camera based on a position of the foreground object [Hong: ¶ [0122]: In such example embodiments, the electronic device 201 may crop the sharpest portions of the images captured by the first camera 253a and the images captured by the second camera 253b. The cropped portions are then combined to produce an extended depth of field image. For example, the electronic device 201 may crop a subject of the images captured by the first camera 253a at a short focus distance and a background of the images captured by the second camera 253b at a long focus distance. The cropped subject and background are then combined to produce an extended depth of field image that is sharper than the 2-D images captured by the cameras 253a, 253b].
Regarding Claims 5, 12, and 18, Benchemsi discloses all the limitations of Claims 1, 9, and 15, respectively, and is analyzed as previously discussed with respect to those claims.
Benchemsi may not explicitly disclose further comprising modifying an amount of optical blur in the output video stream by adjusting a focus of the second camera.
However, Hong discloses further comprising modifying an amount of optical blur in the output video stream by adjusting a focus of the second camera [Hong: ¶ [0140]].
Regarding Claims 6 and 19, Benchemsi discloses all the limitations of Claims 1, 9, and 15, respectively, and is analyzed as previously discussed with respect to those claims.
Benchemsi may not explicitly disclose wherein a first field of view of the first camera and a second field of view of the second camera at least partially overlap.
However, Hong discloses wherein a first field of view of the first camera and a second field of view of the second camera at least partially overlap [Hong: ¶ [0135]: portions of the captured 2-D images that do not overlap may be cropped and removed. That is, only the overlapping portions of the captured 2-D images are combined to produce a high dynamic range image. In such example embodiments, the high dynamic range image may have a different panoramic scope than the captured 2-D images].
Regarding Claims 7, 13, and 20, Benchemsi discloses all the limitations of Claims 1, 9, and 15, respectively, and is analyzed as previously discussed with respect to those claims.
Benchemsi may not explicitly disclose further comprising replacing one or more edge artifacts around the foreground object in the output video stream using the live artificial background image.
However, Hong discloses further comprising replacing one or more edge artifacts around the foreground object in the output video stream using the live artificial background image [Hong: ¶ [0147] In at least some example embodiments, after having identified the boundaries associated with a subject from an in-focus image, the electronic device 201 may crop the subject along its identified boundaries. The electronic device 201 may then combine the cropped in-focus subject and a cropped out-of-focus background to produce a background out-of-focus image. In at least some such example embodiments, the electronic device 201 may crop the out-of-focus background based on the identified boundaries of the subject. That is, the electronic device 201 may use the identified boundaries of the subjects to the background from the out-of-focus images. The cropped in-focus subject and out-focus background are then combined to produce a background out-of-focus image; and ¶ [0148]: In at least some embodiments, 408 (and 802) may only be performed if one or more predetermined criteria are met, for example, if the subject in the 2-D images is determined (for example, in the manner described above with reference to 502) to be far enough away from the cameras 253a, 253b (i.e. beyond a pre-defined threshold distance) for the merging to produce good results].
Claim(s) 8 and 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Benchemsi as applied to claims 1 and 9 above, and further in view of Prentice et al. (US 2012/0327266 A1)
Regarding Claims 8 and 14, Benchemsi discloses all the limitations of Claims 1, 9, and 15, respectively, and is analyzed as previously discussed with respect to those claims.
Benchemsi may not explicitly disclose wherein detecting the foreground object is performed using at least one of an artificial intelligence algorithm or an image processing algorithm.
However, Prentice discloses wherein detecting the foreground object is performed using at least one of an artificial intelligence algorithm or an image processing algorithm [Prentice: ¶ [0084]: The main subject can be the subject identified by the auto-focus algorithm. Alternately, the evaluation image 215 can be analyzed with a main subject detection algorithm. Any main subject detection algorithm known in the art can be used in accordance with the present invention].
It would have been obvious to one having ordinary skill in the art before the effective filing date to combine the object detection method of Prentice with the processing of Benchemsi in order to provide improved object detection methods.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN R MESSMORE whose telephone number is (571)272-2773. The examiner can normally be reached Monday-Friday 9-5 EST/EDT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chris Kelley can be reached at 571-272-7331. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONATHAN R MESSMORE/Primary Examiner, Art Unit 2482
Prosecution Insights