Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Applicant’s amendments filed on 08/05/2026 have been received and considered. Claims 1-20 are pending. Claims 1, 11, and 19 has been amended. No new claims have been added or cancelled.
Claim Objections
The objection to claim 1 due to minor informalities are withdrawn in view of Applicant’s amendments to claim 1.
Response to Arguments
Applicant’s arguments, see pg. 7, filed 08/05/2026, with respect to the rejections of claims 1, 11, and 19 under 35 U.S.C. 102 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, new ground of rejections is made in view of Imamura (JP 2011139300 A).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-7, 9, 11-15, 17, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Tsau in view of Imamura (JP 2011139300 A, see English translation).
Regarding claim 1, Tsau teaches a computing device for compositing videos, the computing device comprising: ([col. 1, lines 7-10] “The present invention relates generally to a mobile computing device, and more specifically to a mobile phone or smartphone that can produce a video-in-video (ViV) video stream having a three layer video scene.”)
a first camera configured to capture a user video; ([col. 2, lines 59-62] “Since the front camera faces the user of the mobile computing device, the front camera video stream includes a subject, which is the user of the mobile computing device.”. Note: the “front camera video stream” is mapped to the user video.)
and processing circuitry configured to, receive a background video, wherein the background video is received in real time; ([col. 3, lines 1-3] “In one example, the back cameras of the mobile computing device face the view that the user of the mobile computing device also faces.”, where “Thus, in one example, a live view of an ongoing event can be broadcasted in real-time from a mobile computing device with the user of the mobile computing device incorporated into the live view. In reality, the presenter is not in the live view.” [col. 3, lines 17-21], and “The resulting ViV video stream shows the subject integrated into the background scene that is actually viewed by the subject” [col. 3, lines 12-14]. Note: the live view viewed by the user recorded by the back cameras is mapped to a background video.)
select a target object from the user video; ([col. 4, lines 47-52] “As shown in the depicted example, the video processor 400 is coupled to identify a subject in a video scene in accordance with the teachings of the present invention. In the example, a front camera video stream 204 is coupled to be received from the front camera of a mobile computing device by a face detection unit 402 of video processor 400.”. Note: the “subject in a video scene” is mapped to the target object.)
generate a converted video based on the target object and the user video; ([col. 2, lines 62-67] “In one example, the subject can be extracted by trimming the background such that the front camera video stream include the face and body of the subject alone with the background removed from the scene. In one example, the front camera video stream including the subject alone is properly scaled to form a mid video layer scene.” Note: the front camera video stream of the subject without the background is mapped to the converted video.)
generate a composited video based on the background video and the converted video; ([col. 3, lines 8-14] “In the example, the mid video layer scene from the front camera is overlaid on the top of the back video layer scene, and the front video layer scene is overlaid on the top of the mid video layer scene, forming a ViV video stream having three video layers. The resulting ViV video stream shows the subject integrated into the background scene that is actually viewed by the subject”, where “objects in a scene of the back camera video stream produced by a back camera can be divided into a far zone and a near zone.” [col. 3, lines 3-5]. Note: the ViV video stream is mapped to a composited video.)
and output the composited video to a display device ([col. 3, lines 15-17] “In one example, the formed ViV video stream can be uploaded to a server and then broadcast by the server in real-time through the Internet.”, where “In one example, video stream Video 1 is a real-time video stream captured by mobile computing device 802.” [col. 7, lines 46-47], and “The receivers of the broadcasted video stream Video 1 may include another mobile computing device 810, such as for example a mobile phone, a smartphone, a tablet computer, or the like. In this manner, the user of mobile computing device 810 may watch video stream Video 1.” [col. 8, lines 4-9]).
Tsau fails to teach adjust a setting of the background video based on an equivalent setting of the user video. However, this is known in the art as taught by Imamura.
Imamura teaches adjust a setting of the background video based on an equivalent setting of the user video ([0041] “In this case, if the first frame rate of the background video to be composited matches the second frame rate of the foreground video to be composited, a suitable composite video can be generated by compositing the background video and the foreground video as they are.”. Note: the “frame rate” is mapped to “setting”, where settings may include “may adjust the frame rate of the background video BGV based on the frame rate of the user video UV” (see Applicant’s specifications [0038])).
Imamura is analogous to the claimed invention, as both relate to compositing videos. Imamura further teaches that “if the first frame rate of the background video to be composited and the second frame rate of the foreground video to be composited do not match, compositing the background video and foreground video as they are will not produce a proper composite video. For example, if you play the video at the same speed as the one with the higher frame rate, the resulting composite video will have the subject in the lower-frame-rate video appearing to move too quickly during playback.” [0042]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Imamura to Tsau in order to create a proper composite video where the foreground and background videos match in speed.
Regarding claim 2, the combination of Tsau and Imamura teaches the computing device of claim 1, wherein the background video is a real time video received in real time; (Tsau; [col. 3, lines 1-3] “In one example, the back cameras of the mobile computing device face the view that the user of the mobile computing device also faces.”, where “Thus, in one example, a live view of an ongoing event can be broadcasted in real-time from a mobile computing device with the user of the mobile computing device incorporated into the live view. In reality, the presenter is not in the live view.” [col. 3, lines 17-21], and “The resulting ViV video stream shows the subject integrated into the background scene that is actually viewed by the subject” [col. 3, lines 12-14])
and the processing circuitry is further configured to, convert the user video in real time; (Tsau; [col. 3, lines 17-23] “Thus, in one example, a live view of an ongoing event can be broadcasted in real-time from a mobile computing device with the user of the mobile computing device incorporated into the live view. In reality, the presenter is not in the live view. Since the presenter actually views the live view, the presenter can comment precisely on the live view.”)
and generate the composited video based on the real time background video and the real time converted video (Tsau; [col. 3, lines 12-17] “The resulting ViV video stream shows the subject integrated into the background scene that is actually viewed by the subject, with proper occlusion. In reality, the subject is not in the background. In one example, the formed ViV video stream can be uploaded to a server and then broadcast by the server in real-time through the Internet.”)
Regarding claim 3, the combination of Tsau and Imamura teaches the computing device of claim 1, wherein the background video is a real time video transmitted by an external server. (Tsau; [col. 9, lines 7-16] “FIG. 10 shows an example in which the background stereo scenes 106 and 108 are not from mobile computing device 802 in accordance with the teachings of the present invention. To illustrate, a mobile computing device 1000, or another device having a stereo camera, including a left camera 1002 producing a left video stream including a left video scene 106 of FIG. 1, and a right camera 1004 producing a right video stream including a right video scene 108 of FIG. 1, uploads 1006 a stereo video stream Video 2 including the left and right video streams to server 804 in a network 800.”, where “Mobile computing device 802 receives 1008 stereo video stream Video 2 from server 804. A video stream including video stream 204 of FIG. 2 produced by front camera 906 of mobile computing device 802 is processed together with video stream Video 2 forming a ViV video stream Video 1 having three video layers as described previously in accordance with the teachings of the present invention.” [col. 9, lines 20-27], and “In one example, the uploading and broadcasting may be real-time.” [col. 9, lines 29-30]).
Regarding claim 4, the combination of Tsau and Imamura teaches the computing device of claim 1, further comprising: a second camera configured to capture the background video in real time. (Tsau; [col. 3, lines 1-3] “In one example, the back cameras of the mobile computing device face the view that the user of the mobile computing device also faces.”, where “the background scene that is actually viewed by the subject” [col. 3, lines 13-14]).
Regarding claim 5, the combination of Tsau and Imamura teaches the computing device of claim 1, further comprising: memory configured to store the background video (Tsau; [col. 1, lines 12-17] “A video stream is uploaded from a stored video file. The stored video file may be produced by a single video camera.”, where “The video processor included in mobile computing device 802 uploads 806 a video stream Video 1 to server 804. In one example, video stream Video 1 is a real-time video stream captured by mobile computing device 802 . In one example, video stream Video 1 may be a recorded video stream, which is previously stored in mobile computing device 802.” [col. 7, lines 44-50]).
Regarding claim 6, the combination of Tsau and Imamura teaches the computing device of claim 1, wherein the processing circuitry is further configured to: execute at least one application, wherein the background video is a video obtained from the at least one application (Tsau; [col. 3, liens 1-7] “In one example, the back cameras of the mobile computing device face the view that the user of the mobile computing device also faces. Based on triangulation, objects in a scene of the back camera video stream produced by a back camera can be divided into a far zone and a near zone. Objects in the far zone are included in a back video layer scene, and objects in the near zone are included in a front video layer scene.”, where “As mentioned previously, the processes explained above may be implemented with the utilization of computer software and hardware. The techniques described may constitute machine-executable instructions embodied within a tangible or non-transitory machine (e.g., computer) readable storage medium, that when executed by a machine, such as for example video processor 400, will cause the machine to perform the operations described.” [col. 9, lines 40-47]).
Regarding claim 7, the combination of Tsau and Imamura teaches the computing device of claim 1, wherein the processing circuitry is further configured to generate the converted video by: converting, using a Chroma key method, at least one pixel other than a pixel corresponding to the target object of the user video (Tsau; [col. 7, lines 3-6] “In process block 712, a subject in a single camera scene, such as for example the front camera, is identified using chroma key compositing methods and/or a video processor to detect the face and body of the subject.”, where “For instance, in one example, a green screen or a blue screen technique may be utilized. The green screen or blue screen techniques, which are also known as chroma key compositing or chroma keying, are a special technique for composing or layering two images together based on color hues or chroma range. The technique has been used heavily in many fields to remove a background from the subject of a photo or video... No part of the subject in the video may duplicate a color used in the background. In this manner, the subject stands in front of a green or blue screen. The background of green or blue screen is trimmed in a background trimming process.” [col. 4, lines 1-8]).
Regarding claim 9, the combination of Tsau and Imamura teaches the computing device of claim 1, wherein the processing circuitry is further configured to: transmit the composited video to an external device or an external server (Tsau; [col. 7, lines 44-48] “The video processor included in mobile computing device 802 uploads 806 a video stream Video 1 to server 804. In one example, video stream Video 1 is a real-time video stream captured by mobile computing device 802.”, where “In the example depicted in FIG. 8, the generated output ViV video stream is shown as Video 1, which is uploaded 806 to server 804.” [col. 8, lines 50-52], and “In the illustrated example, server 804 broadcasts video stream Video 1 to individual receivers, such as for example receivers 810, 812, 814, and 816. The receivers of the broadcasted video stream Video 1 may include another mobile computing device 810, such as for example a mobile phone, a smartphone, a tablet computer, or the like.” [col. 8, lines 2-7]).
Regarding claim 11, claim 11 recites substantially similar limitations to claim 1, but in a method form. The combination of Tsau and Imamura further teaches a video compositing method (Tsau; [col. 1, lines 53-56] “FIG. 7 shows an example block diagram summarizing the process to produce a ViV video stream having three layers of video scene in accordance with the teachings of the present invention.”).
Regarding claim 12, claim 12 recites substantially similar limitations to claim 3, therefore, will be rejected under the same rationale as claim 3.
Regarding claim 13, claim 13 recites substantially similar limitations to claim 4, therefore, will be rejected under the same rationale as claim 4.
Regarding claim 14, the combination of Tsau and Imamura teaches the video compositing method of claim 11, wherein the receiving of the background video comprises: receiving a previously-recorded video (Tsau; [col. 7, lines 44-50] “The video processor included in mobile computing device 802 uploads 806 a video stream Video 1 to server 804... In one example, video stream Video 1 may be a recorded video stream, which is previously stored in mobile computing device 802.”).
Regarding claim 15, claim 15 recites substantially similar limitations to claim 6, therefore, will be rejected under the same rationale as claim 6.
Regarding claim 17, claim 17 recites substantially similar limitations to claim 7, therefore, will be rejected under the same rationale as claim 7.
Regarding claim 19, claim 19 recites substantially similar limitations to claim 1, but in a system form. The combination of Tsau and Imamura further teaches a video compositing system comprising: (Tsau; [col. 9, lines 40-42] “As mentioned previously, the processes explained above may be implemented with the utilization of computer software and hardware.”)
processing circuitry; and memory connected to the processing circuitry and configured to store computer readable instructions, wherein the processing circuitry, by executing the computer readable instructions, is caused to, ([col. 9, lines 42-47] “The techniques described may constitute machine-executable instructions embodied within a tangible or non-transitory machine (e.g., computer) readable storage medium, that when executed by a machine, such as for example video processor 400, will cause the machine to perform the operations described.”).
Claims 8, 10, 16, 18, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Tsau in view of Imamura, and further in view of Do et al. (US 2011/0242277 A1, hereinafter Do).
Regarding claim 8, the combination of Tsau and Imamura teaches the computing device of claim 1, but fails to teach wherein the processing circuitry is further configured to generate the converted video by: adjusting alpha values of at least one pixel other than a pixel corresponding to the target object of the user video. However, this is known in the art as taught by Do.
Do teaches wherein the processing circuitry is further configured to generate the converted video by: adjusting alpha values of at least one pixel other than a pixel corresponding to the target object of the user video ([0058] “At block 790, an alpha mask may be generated to convert the foreground categorized pixels to a 0xFF alpha value and convert other categorized pixels to a 0x00 alpha value.”, where “FIG. 7 illustrates a flow diagram of a method 700 for color based processing for the identification of a foreground portion and a background portion to create a foreground video.” [0048]).
Do is analogous to the claimed invention, as both relate to compositing a background and foreground video. Do further teaches that their embodiment would “provide a more robust video experience allowing greater user control or interaction with the second background or virtual objects comprised within the second background.” [0007]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Do to Tsau in order to allow for better user control when compositing two live videos.
Regarding claim 10, the combination of Tsau, Imamura, and Do teaches the computing device of claim 1, wherein the processing circuitry is further configured to: select an object included in the user video as the target object, the object selected via a user input. (It is note that Tsau fails to explicitly teach the object is selected via a user input. However, it is well known in the art that any mobile computing devices have different type of input device, such as, touch screen, microphone, keyboard, mouse/trackpad, etc. in order to interact with the mobile device. In addition, Do explicitly teaches [0032] “Regardless of whether the input control is received from a user gesture, mouse control, and/or remote control, the input control may be used to control where to place, insert, and/or embed at least one foreground video into the new background provided by the background feed. Moreover, in some embodiments, the control input may be used to select a specific foreground video.”).
Do is analogous to the claimed invention, as both relate to compositing a background and foreground video. Do further teaches that their embodiment would “provide a more robust video experience allowing greater user control or interaction with the second background or virtual objects comprised within the second background.” [0007]. Therefore, it would be obvious for one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teachings of Do to Tsau in order to allow for better user control when compositing two live videos.
Regarding claim 16, claim 16 recites substantially similar limitations to claim 10, therefore, will be rejected under the same rationale as claim 10.
Regarding claim 18, claim 18 recites substantially similar limitations to claim 8, therefore, will be rejected under the same rationale as claim 8.
Regarding claim 20, claim 20 recites substantially similar limitations to claims 3 and 8, therefore, will be rejected under the same rationale as claims 3 and 8.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALICIA HA whose telephone number is (571)272-3601. The examiner can normally be reached Mon-Thurs 9:30 AM - 6:30 PM, and Fri 9:30 AM - 1:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (571) 272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ALICIA HA/Examiner, Art Unit 2611
/KEE M TUNG/Supervisory Patent Examiner, Art Unit 2611
Prosecution Insights