DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Elmoznino et al. (US 2021/0150684 A1 – hereinafter Elmoznino) and Patterson et al. (US 2021/0272599 A1 – hereinafter Patterson).
Regarding claim 1, Elmoznino discloses a method for processing media content, comprising: obtaining first media content ([0042] – receiving images or a video at a target device); applying a first effect to the first media content by a first model to generate second media content ([0042] – applying a first effect by a run-time model to translate the inputted images or video to a second domain space, i.e. with applied effects to output as augmented reality, virtual reality or mixed reality); and providing the second media content ([0042] – providing the second media content as augmented reality, virtual reality or mixed reality), wherein the first model is trained by: training a model associated with the first effect with a first sample set to determine a reference model (Fig. 1; [0032]-[0034] – training a first model associated with the first effect with sample set 102 to determine a reference model, which is the first model after trained as further described at least in [0035]); constructing a second sample set based on processing results for a plurality of sample images from the reference model ([0035]; [0041]; [0065] – constructing a second sample set, i.e. sample set 106, based on filtering of a plurality of sample images from the trained first model, which is the reference model); and training the first model with the second sample set ([0035]-[0036]; [0041] – training the run-time model with the second sample set as ground truths).
However, Elmoznino does not disclose the model as a pre-trained model.
Patterson discloses a model as a pre-trained model ([0055]).
One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to incorporate using a pre-trained model into the model in the method taught by Elmoznino above because using a pre-trained model instead of training a model from scratch would have saving time and computational resources (pre-trained models can be developed on robust systems and distributed to mobile devices with less processing power).
Regarding claim 2, Elmoznino in view of Patterson also discloses the method of claim 1, wherein the plurality of sample images comprises a real image and a composite image ([0035]; Fig. 1 – a real image in first domain space and a composite image in second domain space).
Regarding claim 3, Elmoznino in view of Patterson also discloses the method of claim 1, wherein constructing the second sample set based on the processing results of the reference model for the plurality of sample images comprises: processing the plurality of sample images by the reference model to generate a plurality of first images corresponding to the plurality of sample images ([0035]; [0041] – by generating a plurality of paired images from the trained first model 104, and constructing the second sample set based on filtering the plurality of the paired images); and constructing the second sample set based on the plurality of first images ([0035]; [0041] – by generating a plurality of paired images from the trained first model 104, and constructing the second sample set based on filtering the plurality of the paired images).
Regarding claim 4, Elmoznino in view of Patterson also discloses the method of claim 3, wherein constructing the second sample set based on the plurality of first images comprises: filtering out at least one image not satisfying a predetermined condition from the plurality of first images to obtain a plurality of second images ([0038]-[0039] – filtering pairs of images from the output of the first model 104 using a quality measure to obtain a plurality of second images used to train the run-time model 112); and constructing the second sample set based on the plurality of second images and corresponding sample images ([0065] - filtering the plurality of paired images before providing to train the run-time model).
Regarding 5, see the teachings of Elmoznino in view of Patterson as discussed in claim 4 above. However, Elmoznino and Patterson do not disclose filtering out the at least one image not satisfying the predetermined condition from the plurality of first images comprises: detecting, in each of the plurality of first images, a set of feature points associated with a predetermined object; and filtering out the at least one image not satisfying the predetermined condition from the plurality of first images based on a plurality of sets of the feature points, the predetermined condition being related to at least one of: a number of the set of feature points or a position relationship of the set of feature points.
Official Notice is taken that (1) selecting high-quality images to be used as a training data set comprising (i) detecting, for each candidate image, a set of feature points associated with a predetermined object such as a human face, etc.; and (ii) removing images that do not have enough feature points for the object, e.g. insufficient number of feature points that identify a human face, is well known in the art.
One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to incorporate the filtering of images based on a number of feature points above into the method taught by Elmoznino and Patterson to improve the accuracy of the model.
Regarding claim 6, Elmoznino in view of Patterson also discloses the method of claim 3, wherein constructing the second sample set based on the plurality of first images comprises: determining, based on a predetermined application range of the first effect, a set of first image regions in the plurality of first images that is independent of the first effect ([0035]-[0036]; [0040] – determining regions in the images in the first domain where a visual effect is to be applied, e.g. skin regions, facial regions, etc.); adjusting the set of first image regions in the plurality of first images based on the plurality of sample images ([0035]-[0036]; [0040] – adjusting the image regions in the images of the first domain to provide images in the second domain); and constructing the second sample set based on the adjusted plurality of first images ([0035]-[0036]; [0041] – using the images as the second sample set to train the run-time model).
Regarding claim 7, Elmoznino in view of Patterson also discloses the method of claim 6, wherein adjusting the set of first image regions in the plurality of first images based on the plurality of sample images comprises: determining a set of second image regions on a sample image associated with the first image, the set of second image regions being associated with the set of first image regions ([0035]-[0036]; [0040] – determining regions in the images in the second domain, i.e. regions containing the applied visual effect); and replacing the set of first image regions with the set of second image regions ([0040] – applying the effect into the skin regions, facial regions of the images in the first domain).
Regarding claim 8, Elmoznino in view of Patterson also discloses the method of claim 6, wherein adjusting the set of first image regions in the plurality of first images based on the plurality of sample images comprises: determining a set of third image regions on a sample image associated with the first image, the set of third image regions being associated with the set of first image regions ([0040] – for a first region as a skin region, a third region is a small region in the skin region where an effect is to be applied, e.g. a tattoo); and adjusting attribute information of the set of first image regions based on attribute information of the set of third image regions ([0040] – adjusting pixel values of the skin region at the small region where the effect is to be applied, e.g. a tattoo).
Regarding claim 9, Elmoznino in view of Pattern also discloses method of claim 1, wherein the pre-trained model is a first pre-trained model (Fig. 1; [0032]-[0034] – the first model 104 before fully trained is a first pre-trained model according to a combination with Patterson as discussed in claim 1 above).
However, such a proposed method does not comprise the feature of “training the first model comprises: training a second pre-trained model associated with the first effect with the second sample set, to obtain the first model.”
Patterson discloses a model as a pre-trained model ([0055]).
One of ordinary skill in the art before the effective filing date of the claimed invention would have been motivated to further incorporate using a pre-trained model into the run-time model as for the model in the method taught by Elmoznino and Patterson above because using a pre-trained run-time model instead of training a model from scratch would have saving time and computational resources (pre-trained models can be developed on robust systems and distributed to mobile devices with less processing power).
Claim 10 is rejected for the same reason as discussed in claim 1 above in view of Elmoznino also disclosing an electronic device (Fig. 7; [0031] – device 700), comprising: at least one processor ([0062]; [0064]-[0065]; Fig. 7 – one or more processors 702); and at least one memory, coupled to the at least one processor and storing instructions for execution by the at least one processor (Fig. 7; [0064] – storage device 712), wherein the instructions, when executed by the at least one processor, cause the electronic device to perform the recited acts ([0064] – also see discussion of claim 1 above).
Claim 11 is rejected for the same reason as discussed in claim 2 above.
Claim 12 is rejected for the same reason as discussed in claim 3 above.
Claim 13 is rejected for the same reason as discussed in claim 4 above.
Claim 14 is rejected for the same reason as discussed in claim 5 above.
Claim 15 is rejected for the same reason as discussed in claim 6 above.
Claim 16 is rejected for the same reason as discussed in claim 7 above.
Claim 17 is rejected for the same reason as discussed in claim 8 above.
Claim 18 is rejected for the same reason as discussed in claim 9 above.
Claim 19 is rejected for the same reason as discussed in claim 1 above in view of Elmoznino also disclosing a non-transitory computer-readable storage medium having a computer program stored thereon (Fig. 7; [0064] – storage device 712 storing a computer program), wherein the computer program is executable by a processor ([0062]; [0064]-[0065]; Fig. 7 – executable by one or more processors 702) to perform the recited acts ([0064] – also see discussion of claim 1 above).
Claim 20 is rejected for the same reason as discussed in claim 2 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HUNG Q DANG whose telephone number is (571)270-1116. The examiner can normally be reached IFT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Thai Q Tran can be reached at 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HUNG Q DANG/Primary Examiner, Art Unit 2484
