Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Allowable Subject Matter
Claims 4, 6-11, 25, 27-29 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5, 20-21, 23-24, 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cao U.S. Patent/PG Publication 20190325631 in view of Ye U.S. Patent/PG Publication 9224248.
Regarding claim 1 (independent):
An image rendering processing method, comprising: (Cao [0080] The machine 1200 may include processors 1204, memory memory/storage 1206, and I/O components 1218, which may be configured to communicate with each other such as via a bus 1202. The memory/storage 1206 may include a memory 1214, such as a main memory, or other memory storage, and a storage unit 1216, both accessible to the processors 1204 such as via the bus 1202. The storage unit 1216 and memory 1214 store the instructions 1210 embodying any one or more of the methodologies or functions described herein. The instructions 1210 may also reside, completely or partially, within the memory 1214, within the storage unit 1216, within at least one of the processors 1204 (e.g., within the processor's cache memory), or any suitable combination thereof, during execution thereof by the machine 1200. Accordingly, the memory 1214, the storage unit 1216, and the memory of processors 1204 are examples of machine-readable media.).
obtaining, in response to an effect display request for a target object in an original image, (Cao [0046] At operation 402, the presentation module 306 causes display of a presentation of image data at a client device (e.g., client device 102A), wherein the image data depicts a set of facial landmarks of a user. For example, the image data may be collected by a camera associated with the client device 102A, such as a front facing camera.)(Cao [0055] In some embodiments, a user of the client device 102A explicitly provides the expression definitions via one or more user inputs. For example, a user may form their face into a particular expression and capture an image that depicts the expression at the client device 102A.)
a three-dimensional model of the target object (Cao [0047] At operation 404, the modeling module 304 generates a 3D model (e.g., a 3D bitmoji) based on the set of facial landmarks, in response to the presentation module 306 displaying the presentation of the image data. In some embodiments, the modeling module 304 generates the 3D model in response to detecting the presence of facial landmarks within the image data.).
generating a two-dimensional texture image corresponding to the target object based on a three-dimensional model of the target object and a target texture image (Cao [0021] For example, display instructions may include a 3D graphical element (e.g., a balloon, confetti, hearts) to be presented within the presentation of the image data in response to identifying a particular expression, or by exaggerating one or more facial landmarks to emphasize an expression (e.g., drop jaw, eyes pop out of head, eyes change to stylized “X”). The user may additionally be able to interact with the 3D graphical elements by providing further inputs as movements of facial landmarks (e.g., winking and eye, puckering or blowing, etc.), or as tactile inputs into a touch enabled device.)(Cao [0056] At operation 504, the modeling module 304 selects a graphical element from among a set of graphical elements based on the expression identified by the facial tracking module 302. For example, responsive to the facial tracking module 302 identifying an expression, the modeling module 304 retrieves the corresponding expression definition which includes instructions to display a particular graphical element, such as a 3D object, along with the presentation of the image data at the client device 102A.).
rendering the two-dimensional texture image by a target material to generate a two-dimensional target image (Cao [0058] At operation 510 the presentation module 306 applies a change to the graphical element based on attributes of the second facial tracking input, such as an input value. [0059] For example, the graphical element displayed within the presentation of the image data may include a 3D balloon displayed proximate to a region of the 3D bitmoji that represents the user's lips. The user may pucker their lips to indicate that they are blowing up the balloon. In response to detecting the movement of the user's lips, the presentation module 306 causes the 3D balloon to inflate as if the user was inflating it.)(Cao [0040] xamples of audio and visual content include pictures, texts, logos, animations, and sound effects, as well as animated facial models, such as those generated by the augmented expression system 124. An example of a visual effect includes color overlaying. The audio and visual content or the visual effects can be applied to a media content item (e.g., a photo) at the client device 102. For example, the media overlay including text that can be overlaid on top of a photograph generated taken by the client device 102.).
and coinciding the two-dimensional target image with the original image to display an effect image of the target object (Cao [0061] At operation 602, the communication module 308 generates a flattened presentation based on the image data and the 3D bitmoji. The flattened presentation comprises a merging of the image data and the 3D bitmoji. After merging the image data and the 3D bitmoji into a single layer, at operation 604 the user of the client device 102A assigns the flattened presentation to a message. For example, the message may be addressed to one or more recipients, including a second client device 102B.)
Cao discloses material as describe above. However, for the purposes of compact prosecution and for further clarity, in a related field of endeavor, Ye teaches:
rendering the two-dimensional texture image by a target material (Ye C7 L15-40 Referring to FIG. 16, three virtual visual effects that can be applied in each eyelid area, including (a) color series, (b) alpha blending, and (c) superposition. Regarding alpha blending visual effect, it is defined as follow: the input color can possess degrees of transparency from 0% to 100%, in which the degree of transparency can be proportional, for example, using blue color using a virtual makeup input tool, to be inputted to a makeup-allowed area using a virtual makeup input tool will produce 20% translucent blue color (or of 80% transparent), and by drawing or inputting twice using the virtual makeup input tool, it will change to 40% translucent blue color, and can be repeated until 100% translucent/opaque blue color is achieved (or equal to 0% transparent). Regarding superposition, it is defined as follow: color can be mixed or blended together to obtain cumulative effects, for example, by adding green color using a virtual makeup input tool to be inputted to one segment in a makeup-allowed area will produce 20% green color, and by drawing or inputting twice using the virtual makeup input tool, it will change to 40% in green color, and furthermore, by adding red color to be inputted to the same makeup-allowed area segment will be blended and mixed with the 40% in green color to achieved the mixed blended color. Regarding color series, it is defined as follow: each make-up allowed area can possess multiple number of colors at the same time, similar to the different color bands found in a rainbow, and the color of each layer can include alpha blending and superposition effects.)(Ye FIGS. 30a and 30b show virtual makeup applied on a 3D face model using pre-defined adjustable makeup texture package of still further yet another embodiment, in which color overlapping can be produced using a mouse cursor, touch control or the tip of a tracked object.)
Therefore, it would have been obvious before the effective filing date of the claimed invention to adjust overlays as taught by Ye. The motivation for doing so would have been to provide the user with more customization and options. Further, the rationale for doing so would have been that it combines prior art elements according to known methods to yield predictable results where Cao is using AR to overlay graphics on an image of a person, and Ye is using AR to overlay graphics on an image of a person but with additional customization, where the end result for both is outputting an image of a person with overlays. Therefore it would have been obvious to combine Ye with Cao to obtain the invention.
Regarding claim 2:
The image rendering processing method according to claim 1, has all of its limitations taught by Cao in view of Ye. Cao further teaches wherein the obtaining a three-dimensional model of the target object comprises:
obtaining object attributes of the target object in the original image and querying a preset three-dimensional model database to obtain a three-dimensional model corresponding to the object attributes (Cao [0049] In further embodiments, the modeling module 304 generates a 3D bitmoji on the fly, based on the attributes of the facial landmarks, such that the 3D bitmoji resembles the user. For example, the modeling module 304 may access a bitmoji repository that contains bitmoji elements, select a set of bitmoji elements based on the attributes of the facial landmarks, and generate the 3D bitmoji based on the selected bitmoji elements.).
Regarding claim 3:
The image rendering processing method according to claim 1, has all of its limitations taught by Cao in view of Ye. Cao further teaches wherein the obtaining a three-dimensional model of the target object comprises:
tracking a target area of the target object in the original image to generate a three-dimensional model of the target area, wherein the three-dimensional model of the target area comprises: a local three-dimensional model of the target object or a global three-dimensional model of the target object (Cao [0049] In further embodiments, the modeling module 304 generates a 3D bitmoji on the fly, based on the attributes of the facial landmarks, such that the 3D bitmoji resembles the user. For example, the modeling module 304 may access a bitmoji repository that contains bitmoji elements, select a set of bitmoji elements based on the attributes of the facial landmarks, and generate the 3D bitmoji based on the selected bitmoji elements.).
Regarding claim 5:
The image rendering processing method according to claim 1, has all of its limitations taught by Cao in view of Ye. Cao further teaches wherein the generating a two-dimensional texture image corresponding to the target object based on a three-dimensional model of the target object and a target texture image comprises:
conforming the three-dimensional model to the target texture image to generate a three-dimensional texture model and mapping the three-dimensional texture model to a target position corresponding to the target object in a screen, so as to draw and generate a two-dimensional texture image according to the target position (Cao [0019] In further embodiments, the system generates the 3D bitmoji based on attributes of the set of facial landmarks of the user. For example, the attributes may include distances and slopes between various facial landmarks, size and shape of facial landmarks, as well as a complexion of the user. The system selects individual components of the 3D bitmoji based on the attributes of the facial landmarks.)(Cao [0051] At operation 406, the presentation module 306 overlays the 3D bitmoji generated by the modeling module 304 at a position on the image data displayed at the client device 102A based on the locations of the set of facial landmarks. For example, the facial tracking module 302 may identify a set of reference features based on the set of facial landmarks and cause the presentation module 306 to overlay a presentation of the 3D bitmoji on the image data at the client device 102A.).
Regarding claim 20 (independent):
The claim is a/an parallel version of claim 1. As such it is rejected under the same teachings.
Regarding claim 21 (independent):
The claim is a/an parallel version of claim 1. As such it is rejected under the same teachings.
Regarding claim 23:
The claim is a/an parallel version of claim 2. As such it is rejected under the same teachings.
Regarding claim 24:
The claim is a/an parallel version of claim 3. As such it is rejected under the same teachings.
Regarding claim 26:
The claim is a/an parallel version of claim 5. As such it is rejected under the same teachings.
Conclusion
For the prior art referenced and the prior art considered pertinent to Applicant’s disclosure but not relied upon, see PTO-892 “Notice of References Cited”.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JASON PRINGLE-PARKER whose telephone number is (571) 272-5690 and e-mail is jason.pringle-parker@uspto.gov. The examiner can normally be reached on 8:30am-5:00pm est Monday-Friday. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, King Poon can be reached on (571) 270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, seehttp://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JASON A PRINGLE-PARKER/
Primary Examiner, Art Unit 2617
Prosecution Insights