DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. 18250631, filed on 04/26/2023.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 06/14/2023 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Objections
Claims 1-5 are objected to because of the following informalities:
Claims 1-5 recite “METHOD FOR DETECTING AND SEGMENTING THE LIP REGION", which should be lowercase and not within quotation marks.
Claims 1-5 are written as hyphenated steps. The steps should not be separated by hyphens.
Claim 4 recites “used a SLIC algorithm”. In this limitation, “used” should recite “using” and SLIC should be fully written out.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “processing module” in claims 1-4.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-5 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites, “defining and indicating the labelled lip images with algorithms for recognizing patterns for said lip images to be learnt and segmented by said processing module;”. This limitation lacks overall clarity as currently worded. This is being interpreted as, pattern recognition algorithms are performed in the labelled lip images, which are subsequently used for training a segmentation model.
Claim 1 recites “generate a mathematical pattern for a lip product application system”. There is insufficient antecedent basis for this limitation in the claim. This is being interpreted as “generate a mathematical pattern for the lip product application system”.
Claims 2-5 are rejected as dependent on claim 1.
Claim 2 recites the following plurality of antecedence issues (bolded): “- the step of training a machine learning model in said processing module comprises carrying out the encoding and decoding of original lip images received in grayscale and a mask as input in the convolutional network (CNN) U-Net during the training process, generating a predicted mask, and at the end of the training generating a mathematical mask prediction model from the lip images used in the training.” The bolded elements are being interpreted as: the machine learning model; encoding and decoding; the step of training; an end.
Claim 3 recites “the lips or part of the lips” and “the region of the images”. There is insufficient antecedent basis for these limitations in the claim. They are being interpreted as new elements.
Claim 4 recites “the images containing the lip region or part of the lips”, “the images cut out from the image database”, and “the superpixel algorithm”. There is insufficient antecedent basis for these limitations in the claim. They are being interpreted as new elements.
Claim 5 recites the following plurality of antecedence issues (bolded): “The "METHOD FOR DETECTING AND SEGMENTING THE LIP REGION" according to claim 1, characterized by:
- submitting, in a pre-processing step, an original input lip image to the step of segmentation of the image by superpixel with the extraction of contours resulting in the image with the separation between lip and facial skin;
- extracting a mask relating to the image with the separation between lip and facial skin, inserting the information from this mask in the original image, and converting the color space of the original input image from RGB to HSV;
- inserting the mask information in the luminance V channel so as to highlight the separation between lip and facial skin in the final RGB image;
- converting the image in the HSV color space to the RGB color space, obtaining a resulting image;
- inserting the resulting image in the training process using the R-CNN Mask algorithm; and
- carrying out the segmentation training using the R-CNN Mask algorithm with the training base image of part of the lips resulting from the pre-processing step; and
- generating a segmentation model.” The bolded limitations are being interpreted as new elements.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fu (US 20190289986 A1).
Regarding claim 1, Fu, in a first embodiment, teaches, “A "METHOD FOR DETECTING AND SEGMENTING THE LIP REGION", characterized by comprising the steps of: - recognizing patterns by extracting input features from lip images” (Fu, Paragraph 24, “The invention also preferably includes a method for generating an output effect on an input image having a face, comprising: (a) providing a facial image of a user with facial landmarks; (b) locating the facial landmarks from the facial image of the user, wherein the facial landmarks include a first region, and wherein the landmarks associated with the first region are associated with lips of the facial image having a lip color and the first region includes a lip region; (c) converting the lip region of the image into at least one color channel and detecting and analyzing a light distribution of the lip region; (d) feeding the at least one color channel into histogram matching over a varying light distribution to identify a histogram having a pre-defined light distribution that varies from the light distribution of the lip region thereby generating at least one output effect; and (e) combining the output effect with the first image to provide a resultant image having the lip color and the at least one output effect applied to the lip.”)
While the first embodiment of Fu discloses a processing module present in a lip product application system, (Fu, Paragraphs 272 and 2, “Method steps of the techniques described herein can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input image and other related data and generating output. Method steps can also be performed by, and apparatus can be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit). Modules can refer to portions of the computer program and/or the processor/special circuitry that implements that functionality.”; “The present disclosure relates to systems and methods for virtual facial makeup simulation, including methods for virtual removal of makeup and application of makeup and makeup effects to a user image. The present disclosure further relates to systems and methods for virtual facial makeup simulation using a neural network. The present disclosure also relates to various methods and systems for improving virtual facial makeup simulation, including virtual makeup tutorials, makeup recommendations, automatic adjustment of brightness and calibration of color using a color map and standard, a framework of fast facial landmarks detection and tracking and a method of solving the lag problems associated with fast facial movement and the landmark shaking problems associated with a user staying still in a video.”), this embodiment of Fu does not expressly disclose labelling the images for a training base.
A second embodiment of Fu discloses labelling lip images for a training base (Fu, Fig. 9 shows makeup annotation system 5000 which labels images to be used for training in step 4035.)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to store the input images of the first embodiment of Fu as labelled training data in the second embodiment of Fu.
The motivation for doing so would have been to increase the number of training images. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, it would have been obvious to combine the first and second embodiments of Fu to fully disclose, “- recognizing patterns by extracting input features from lip images, labelling them for a training base by means of a processing module present in a lip product application system;”
Fu further discloses, “- defining and indicating the labelled lip images with algorithms for recognizing patterns for said lip images to be learnt and segmented by said processing module;” (Fu, embodiment 1, Paragraph 255, “With reference to 3020 a first frame is identified as an image frame. This may be done using an image pyramid that is generated with different scales. If the current frame has previous landmarks, a face is detected in 3040, and multi-scaled global detector with sliding windows is used to scan the image 3050 to detect the facial landmarks, and, further in 3050, for each region of interest, the Histogram of Gradient is computed and used as the input to the Supported Vector Machine model 3060 to judge which part is the destination region for patch extraction and fitting. See, N. Dalal and B. Triggs, “Histograms of oriented gradients for human detection,” CVPR, pp. 886-893 (2005); and C. Cortes and V. Vapnik, “Support-vector networks,” Machine Learning, pp. 273-297 (1995). Then the windows are down sampled to make the region more accurate in 3070. If a face is not detected, a constrained mean shift 3030 is used in the image pyramid to detect a frame, and if previous landmarks exist, they can be used as the initial shape of the global detector in 3050. If the current frame has previous landmarks, the previous landmarks are used to align to a current bounding box in 3035 as initial landmarks for the Supervised Descent Method (SDM).” Note that as the first and second embodiment of Fu are combined above, the annotated lip images are used for training a face and lip segmentation model that applies makeup to face regions.)
“- training a machine learning model in said processing module with a plurality of exemplified data and their respective answers defining labels that the model should learn and predict to identify and generate a mathematical pattern for a lip product application system.” (Fu, embodiment 1, Figure 9 shows exemplified data (4010) with defining labels (5000a-c) which are used for training mathematical patterns (4035) for the lip product application system.)
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fu in view of Vats (US 20200065559 A1) further in view of Ronneberger (U-Net: Convolutional Networks for Biomedical Image Segmentation).
Regarding claim 2, Fu teaches, “The "METHOD FOR DETECTING AND SEGMENTING THE LIP REGION" according to claim 1,”
While Fu teaches that the step of recognizing patterns by extracting input features from lip images comprises recognizing patterns using a contour prediction model (Fu, embodiment 1, Paragraph 140, “The iterative method is used to detect the lip region using the complexion probability map of the lower part of the face. In each iteration, more offset is added on the base threshold until the binary image contains a contour region that satisfies the above criteria and has the convex hull configuration for the white region. Once such criteria are met, the detected region is considered to be the initial lip region.”), Fu does not disclose infrared images or a convolutional network (CNN) U-Net.
Vats discloses the use of infrared face images (Vats, Paragraph 194, “User interface 1603 may function to allow client device 1612 to interact with a human or non-human user, such as to receive input from a user and to provide output to the user. Thus, user interface 1603 may include input components such as a keypad, keyboard, touch-sensitive or presence-sensitive panel, computer mouse, joystick, microphone, still camera and/or video camera, gesture sensor, tactile based input device. The input component also includes a pointing device such as mouse; a gesture guided input or eye movement or voice command captured by a sensor, an infrared-based sensor; a touch input; input received by changing the positioning/orientation of accelerometer and/or gyroscope and/or magnetometer attached with wearable display or with mobile devices or with moving display; or a command to a virtual assistant.” Paragraphs 6-12 describes the use of facial images.”)
Ronneberger discloses a convolutional network (CNN) U-Net for contour prediction (Ronneberger, Figures 1 and 3 show the CNN U-Net (Fig. 1) that performs contour prediction (Fig. 3c).)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use infrared imaging of Vats for the face images of Fu, and use the CNN U-Net of Ronneberger for the contour prediction of Fu.
The motivation for incorporating Vats would have been to improve image clarity in poorly lit scenes. The motivation for incorporating Ronneberger would have been to improve mouth contour accuracy even with limited annotated data. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, Fu combined with the above teachings of Vats and Ronneberger disclose, “characterized in that: - the step of recognizing patterns by extracting input features from lip images comprises recognizing patterns in infrared images, using a contour prediction model by a convolutional network (CNN) U-Net;”
Fu in view of Vats further in view of Ronneberger further disclose, “and - the step of training a machine learning model in said processing module comprises carrying out the encoding and decoding of original lip images received in grayscale and a mask as input in the convolutional network (CNN) U-Net during the training process, generating a predicted mask, and at the end of the training generating a mathematical mask prediction model from the lip images used in the training.” (As combined above, Figure 1 of Ronneberger shows the encoding and decoding of original images, and Figure 3 of Ronneberger shows an input grayscale images (3a), mask input (3b), and predicted mask (3c) which are used to train the mathematical mask prediction model (3d and output of Fig. 1).)
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fu in view of Li (US 20200342209 A1).
Regarding claim 3, Fu teaches “The "METHOD FOR DETECTING AND SEGMENTING THE LIP REGION" according to claim 1,”
While Fu teaches training an algorithm with training images to extract mouth regions, (Fu, embodiment 1,Paragraph 115 and Figure 3, “In Step 1030, e.g., a landmark detection algorithm may be utilized to locate the fiducial points of the landmarks, through which one can then extract the mouth region and eye region images. A suitable landmark detection software and associated training sets useful herein for this purpose may be found at OpenCV (i.e., opencv.org). Additional software and facial recognition processes such as those of dlib landmark detection (see, http://dlib.net/face_landmark_detection.py.html) and Giaran, Inc. landmark detection may also be used. Many suitable commercial and open-source software exists for facial detection, such as Python, dlib and HOG, as well as for landmark detection and identification of fiducial points, such as that described by V. Kazemi et al., “One Millisecond Face Alignment with an Ensemble of Regression Trees,” KTH, Royal Institute of Technology, Computer Vision and Active Perception Lab, Stockholm, Sweden (2014). Preferred for use herein is Giaran, Inc. software.”), Fu does not expressly disclose the use of a R-CNN Mask algorithm.
Li discloses the use of a R-CNN mask algorithm, (Li, Paragraph 20, “There are a series of frameworks which are flexible and robust for object detection and semantic segmentation like Fast R-CNN[19], Faster R-CNN[20] and Fully Constitutional Network[21]. Faster R-CNN uses a multi-branch design to perform bounding box regression and classification in parallel. Mask-RCNN[22] is an extension of Faster-RCNN, and adds a new branch for predicting segmentation masks based on each Region of Interest. Of particular interest is Mask-RCNN's use of RoIAlign[22], (where RoI in an initialism from the term “Region of Interest”) which allows for significant savings in computation time by taking crops from shared convolutional features. By doing this, it avoids re-computing features for overlapping regions of interest.”)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use the Mask-RCNN of Li for the training algorithm of mouth extraction of Fu.
The motivation for incorporating Li would have been to save computation time and avoid re-computing features, as described above by Li. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, Fu combined with the above teaching of Li disclose, “characterized in that the step of training a machine learning model in said processing module comprises: “- training an R-CNN Mask algorithm with a training image base of the lips or part of the lips in order to learn how to differentiate labial skin from facial skin;”
Fu in view of Li further disclose, “and - generating a segmentation model of the region of the images containing the lip region or part of the lips.” (Fu, embodiment 1, Paragraph 115 and Figure 3, “In Step 1030, e.g., a landmark detection algorithm may be utilized to locate the fiducial points of the landmarks, through which one can then extract the mouth region and eye region images. A suitable landmark detection software and associated training sets useful herein for this purpose may be found at OpenCV (i.e., opencv.org). Additional software and facial recognition processes such as those of dlib landmark detection (see, http://dlib.net/face_landmark_detection.py.html) and Giaran, Inc. landmark detection may also be used. Many suitable commercial and open-source software exists for facial detection, such as Python, dlib and HOG, as well as for landmark detection and identification of fiducial points, such as that described by V. Kazemi et al., “One Millisecond Face Alignment with an Ensemble of Regression Trees,” KTH, Royal Institute of Technology, Computer Vision and Active Perception Lab, Stockholm, Sweden (2014). Preferred for use herein is Giaran, Inc. software.”)
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Fu in view of Achanta (SLIC Superpixels Compared to State-of-the-Art Superpixel Methods).
Regarding claim 4, Fu teaches “The "METHOD FOR DETECTING AND SEGMENTING THE LIP REGION" according to claim 1,”
While Fu teaches an “algorithm being applied to the images cut out from the image database containing the lip region or part of the lips;” and a segmentation model (Fu, embodiment 1, Figure 3 shows facial landmark locating on face images and mouth region extraction, which is further shown in figures 21-23.), Fu does not teach “characterized in that the step of training a machine learning model in said processing module comprises: - grouping the pixels of an image based on the similarity of the color feature by means of a clusterization algorithm that groups elements in a given space of similar features such that determining the learning is pointed out by the group to which it belongs- generating a segmentation model of the region of the images containing the lip region or part of the lips used a SLIC algorithm that performs the clusterization with the k-means method using segment number parameters” or “- changing the hue, saturation and value in the HSV color space, or the RGE color space, so as to render the elements of the image perceptible to the superpixel algorithm.”
Achanta teaches, “- grouping the pixels of an image based on the similarity of the color feature by means of a clusterization algorithm that groups elements in a given space of similar features such that determining the learning is pointed out by the group to which it belongs - generating a segmentation model of the region of the images” “used a SLIC algorithm that performs the clusterization with the k-means method using segment number parameters” and “- changing the hue, saturation and value in the HSV color space, or the RGE color space, so as to render the elements of the image perceptible to the superpixel algorithm.” (Achanta, Section 3.1 and Figure 5:
PNG
media_image1.png
483
1096
media_image1.png
Greyscale
PNG
media_image2.png
212
1056
media_image2.png
Greyscale
PNG
media_image3.png
556
1061
media_image3.png
Greyscale
Accordingly, Achanta teaches the color-based pixel grouping, the segmentation model using a SLIC algorithm (including k-means clustering), and the changing of color values to CIELAB for use in the superpixel algorithm.)
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to use the above described grouping, SLIC segmentation method, and changing of color values of Achanta on the face images of Fu.
The motivation for incorporating Achanta would have been to increase speed and improve segmentation performance, as described in Section 4.3 of Achanta. Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results. Therefore, Fu combined with the above teaching of Achanta disclose, “characterized in that the step of training a machine learning model in said processing module comprises: - grouping the pixels of an image based on the similarity of the color feature by means of a clusterization algorithm that groups elements in a given space of similar features such that determining the learning is pointed out by the group to which it belongs- generating a segmentation model of the region of the images containing the lip region or part of the lips used a SLIC algorithm that performs the clusterization with the k-means method using segment number parameters the algorithm being applied to the images cut out from the image database containing the lip region or part of the lips; and - changing the hue, saturation and value in the HSV color space, or the RGB color space, so as to render the elements of the image perceptible to the superpixel algorithm.”
Allowable Subject Matter
Claim 5 is rejected under 35 USC 112(b) and objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and amended to overcome the 35 USC 112(b) rejection.
The following is a statement of reasons for the indication of allowable subject matter: With respect to claim 5, in addition to other limitations in the claims the Prior Art of Record fails to teach, disclose or render obvious the applicant' s invention as claimed, in particular:
Claim 5 recites: “The "METHOD FOR DETECTING AND SEGMENTING THE LIP REGION" according to claim 1, characterized by: - submitting, in a pre-processing step, an original input lip image to the step of segmentation of the image by superpixel with the extraction of contours resulting in the image with the separation between lip and facial skin; - extracting a mask relating to the image with the separation between lip and facial skin, inserting the information from this mask in the original image, and converting the color space of the original input image from RGB to HSV; - inserting the mask information in the luminance V channel so as to highlight the separation between lip and facial skin in the final RGB image; - converting the image in the HSV color space to the RGB color space, obtaining a resulting image; - inserting the resulting image in the training process using the R-CNN Mask algorithm; and - carrying out the segmentation training using the R-CNN Mask algorithm with the training base image of part of the lips resulting from the pre-processing step; and - generating a segmentation model.”
Fu teaches an automated system for virtual makeup removal and application including facial segmentation steps and machine learning. Vats teaches a video generation method for recognizing neck and body regions. Ronneberger teaches a strategy for biomedical image segmentation using a U-Net CNN. Li discloses a virtual makeup try-on system including facial landmark detection. Achanta teaches a SLIC method of image segmentation. Yan (US20200042769A1) discloses a processing method including face detection and facial region segmentation. However, none of these references disclose the bolded limitations above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON JOSEPH SORRIN whose telephone number is (703)756-1565. The examiner can normally be reached Monday - Friday 9am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON JOSEPH SORRIN/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672