DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in responsive to communication(s): original application filed on 04/19/2024, said application claims a priority filing date of 02/19/2019. Claims 1-19 are pending. Claims 1, 10, and 19 are independent.
Double Patenting
The non-statutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A non-statutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on non-statutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a non-statutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based e-Terminal Disclaimer may be filled out completely online using web-screens. An e-Terminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about e-Terminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-19 are rejected on the ground of non-statutory double patenting as being unpatentable over Claims 7, 2, 4, 3, 5-6, 1, 10, 12, 9, 2, 4, 3, 5-6, 1, 10, 12, and 13 of U.S. Patent No. 11,995,750 in view of O'Hara et al. (US 2018/0047200 A1, pub. date: 02/15/2018), hereinafter O'Hara.
Claim 7 of U.S. Patent No. 11,995,750 contain every limitations of Claim 1 of the instant application except for the limitation of "a face portion and a body portion of the avatar image". Claims 2, 4, 3, 5-6, 1, 10, 12, 9, 2, 4, 3, 5-6, 1, 10, 12, and 13 of U.S. Patent No. 11,995,750 contain every limitations of Claims 2-19 of the instant application respectively except for the limitations of (1) "a face portion and a body portion of the avatar image"; and (2) "perform, in response to receiving a third input to select an object of the at least one object, an edit function of the avatar image corresponding to the selected object". O'Hara teaches both the limitations (1) and (2) in ¶¶ [0059]-[0062] with FIG. 1; ¶¶ [0067]-[0071] with FIGS. 3-4; ¶¶ [0072], [0075]-[0077], [0079]-[0081], and [0083]-[0090] with FIGS. 5-9; and ¶¶ [0087]-[0089] with FIGS.12-13. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention) to include the teaching of O'Hara so that provide more personalization and expression of emotion in animated avatar images (O'Hara, ¶ [0004]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 6-7, 10, 12, 15-16, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over O'Hara et al. (US 2018/0047200 A1, pub. date: 02/15/2018), hereinafter O'Hara in view of DYE et al. (US 2019/0342507 A1, filed on 09/26/2018), hereinafter DYE, Okada et al. (US 10,242,533 B2, filed on 10/14/2016), hereinafter Okada, and Bernstein et al. (US 2017/0212673 A1, pub. date: 07/27/2017), hereinafter Bernstein.
Independent Claims 1, 10, and 19
O'Hara discloses an electronic device (O'Hara, ¶¶ [0057] and [0118]: a mobile device/computer data processing system) comprising: a camera (O'Hara, ¶ [0057]: a mobile device with a front or rear facing camera); a touch screen display (O'Hara, ¶ [0117]: touch screens); memory (O'Hara, ¶ [0117]: tangible memories (e.g. , random access memories (RAMs), read-only memories (ROMs), and/or programmable read only memories (PROMS)), tangible storage devices (e.g., hard disk drives, CD/DVD drives, and/or flash memories)) storing instructions (O'Hara, ¶¶ [0120]-[0121]: the software includes programming instructions may be stored on or in one or more non-transitory, tangible storage devices, such as one or more hard disk drives, CDs, DVDs, and/or flash memories; associated data may be stored in any type of volatile and/or non-volatile memory; the software may be loaded into a non-transitory memory); and a processor (O'Hara, ¶ [0117]: processors); wherein the instructions, when executed by the processor, cause the electronic device to (O'Hara, ¶ [0121]: the software may be loaded into a non-transitory memory and executed by one or more processors; the programming instructions are configured to implement one or more processes and algorithms that implement one or more of the functions of the computer data processing system, as recited herein):
display, on the touch screen display, a preview image comprising a face portion of an avatar image of a user (O'Hara, ¶¶ [0059] and [0061]-[0062] with FIG. 1: before capturing the image of the face, the user may adjust the direction, rotation, zoom, and/or distance of the camera until the preview image of the targeted face is centered within and fills a pre-determined border 101 and the eyes of the face are both on the same horizontal line and centered within an eye level indicator, such as an eye level slot 102; automatically and without user input detect the size, location, and/or rotation of the face in the image and, automatically and without user input, adjust one or more of the same, before the image is captured, so as to cause the image of the face to be centered within and fill the pre-determined border 101 and the eyes of the face to be both on the same horizontal line and centered within the eye level indicator; use any type of image recognition algorithms to make these auto mated adjustments; e.g., detect a face within an image by scanning for different facial features , such as a nose or eyes , by comparing parts of the image to a database of images of facial features , and then by placing a rectangular border around the predicted area of the face using an algorithm to calculate the size of the face in relation to the detected facial feature; then automatically adjust the size and orientation of the detected face to fit within the pre-determined border 101; ¶¶ [0067]-[0068] with FIG.3: allow the user to select and customize a shape of the face of the avatar as shown in FIG. 3; the user may choose a face shape 303 that may be used to generate a border 305 that crops a selected or captured face image 301 after its size, position, and rotation have been adjusted; ¶ [0010]: detecting the size and location of the real face within the photographed image; i.e., detecting a motion of a subject),
display, on the touch screen display, a user interface comprising the avatar image including the face portion and a body portion of the avatar image and at least one object related to performing at least one edit function related to the avatar image in response to receiving a first input (O'Hara, ¶¶ [0058] and [0100]-[0102] with 1902, 1904, and 1905 in FIG. 19 and FIG. 3: after selecting the desired camera, the user may actuate a user control, such as a camera snap button 103, which may activate the selected camera, that may then be used to take a picture of either the user's face or another person's face in in an image capture step 1902; after the user takes a picture using an image capture device in an image capture step 1902, the captured image with the border 305 (i.e., emoji) are shown in FIG. 3, which may be customized in an image transformation step 1904 with further user input, during which features are applied to the captured image, e.g., smoothing, face shape, skin color, and eye color, to transform the selected or captured image to partially resemble a 2D cartoon illustration; the image and 2D illustrated body, collectively referred to herein as the digital avatar, may then open to user customization in a user customization step 1905; ¶¶ [0060] and [0067]-[0071] with FIGS. 3-4: after the image is captured, user-controls is provided for the user to adjust the size, location, and/or rotation of the image of the face with respect to the pre-determined border 101 and the eye level slot 102, so as to cause the image of the face to be centered within and fill the pre-determined border 101 and the eyes of the face to be both on the same horizontal line and centered within the eye level indicator; after the user selects or captures a face image and adjusts its position, size, and/or rotation, the user may choose a face shape 303 that may be used to generate a border 305 that crops a selected or captured face image 301 after its size, position, and rotation have been adjusted; the user can customize the border 305 around the image 301 by widening or narrowing it by dragging one or more border change buttons 302; after completing the selection and customization of a face shape, the user may actuate a user-operated control to step to the next or previous customization option, such as by tapping a forward or reverse arrow button 310; the user may tap the current customization option, such as a "Face shape" 320 label, to call up a menu of customization options and then directly go to the desired option by selecting it from the menu; ¶¶ [0087]-[0089] with FIGS.12-13: allow the user to customize the shape of the body of the avatar as shown in FIG. 12; the user may customize the shape of the body of a digital avatar 1201 underneath the image 301 by adjusting a user-operated control, such as a body size slider 1203 and/or by choosing between two gender options 1204; sliding of the body size slider 2013 may widen or narrow the body of the digital avatar 1201; the male or female gender options 1204 may change the body type of the digital avatar 1201 to reflect either a male or a female shape; the user may choose colors for different articles of clothing worn by the digital avatar 1201 by tapping the color selection button 803; allow the user to customize the color of various articles of clothing worn by the avatar as shown in FIG. 13; pressing the color selection button 803 may open a color selection wheel 901 for the user to select a color for different clothing worn by the digital avatar 1201; the user may select a color from the color selector wheel 901 and then apply the selected color to a shirt on the avatar by tapping a shirt button 1301 , to pants by tapping a pants button 1302 , and to shoes by tapping a shoes button 1303) (O'Hara, ¶¶ [0072], [0075]-[0077], [0079]-[0081], and [0083]-[0090] with FIGS. 5-9: allow the user to customize features of the avatar, such as smoothness and lighting, skin color, hairstyle, and hair color to cause the selected smoothness and lighting, skin color, hairstyle, and hair color to overlay and replace the actual smoothness and lighting, skin color, hairstyle, and hair color, as depicted in the captured image of the real face),
wherein the avatar image displayed in the user interface is the avatar image to which a plurality of animations have been applied, and the applied plurality of animations of the avatar image are displayed at designated time intervals (O'Hara, ¶¶ [0049]-[0054]: creating animated digital avatars, e.g., a digital avatar with animated facial expressions; a computer-generated animation by combining the digital avatar and 2D illustrated images into a collection of frames and by rendering the collection in a timed sequence to create, e.g., a digital avatar with animated facial expressions; ¶¶ [0091]-[0099], [0103], and [0115]-[0117] with FIGS. 14-18, 1906 in FIG. 19, and FIGS. 22-23: various animated avatar previews may be created and presented based on the customization selections made by the user; a grid of animated selectable digital avatar animations may be presented, such as animated avatars 1401, 1403, 1405, and 1407; each animated avatar may present a pre-fabricated sequence of animation frames which may include layers of 2D and 3D animation and optionally text; one or more of these animation frames may be edited to include customizations dictated by the user; editing one or more drawn frames in a pre-determined set of drawn frames to reflect one or more of the customizations that the user specified; user may then signal completion of the selection by tapping a Start Now button 1409; the digital avatar may then be rendered during a render process step 1906; create an animated digital avatar by combining different file types in a render library; create a .plist file 2204, which may include the specifications for the digital avatar, such as, e.g., eye color, skin color, hairstyle, accessory, gender, and body type; translate the .plist file 2204 into a set of animation frames 2205 made up of 2D illustrated images, such as the frames shown in FIGS. 15-18, which may then be rendered in a timed sequence to create an animation image 2206, such as, e.g., a digital avatar with the animated facial expressions 1401, 1403, 1405, and 1407; take in a template for animation, e.g., an .swf file, and user specifications in an accept template; the template for animation may include the data and resources required for rendering a digital avatar into an animation, but may have default features, such as a standard facial image, clothing color, and/or skin color; the template for animation may then be combined with the user specifications to create a file type, e.g., a .plist, which contains both the template for animation and the user specifications in combination, wherein the user specifications may adjust the default features included in the template for animation to reflect the user selections; i.e., user can select one animated avatar from a plurality of animated avatars 1401, 1403, 1405, and 1407 for rendering as shown in FIGS. 15, 16, 17, and 18 respectively with designated time intervals),
receive a second input on (O'Hara, ¶¶ [0091]-[0099], [0103], and [0115]-[0117] with FIGS. 14-18, 1906 in FIG. 19, and FIGS. 22-23: various animated avatar previews may be created and presented based on the customization selections made by the user; a grid of animated selectable digital avatar animations may be presented, such as animated avatars 1401, 1403, 1405, and 1407; each animated avatar may present a pre-fabricated sequence of animation frames which may include layers of 2D and 3D animation and optionally text; one or more of these animation frames may be edited to include customizations dictated by the user; editing one or more drawn frames in a pre-determined set of drawn frames to reflect one or more of the customizations that the user specified; the user may select one of the customized animations, such as by tapping the animation; the user may then signal completion of the selection by tapping a Start Now button 1409; the digital avatar may then be rendered during a render process step 1906; create a .plist file 2204, which may include the specifications for the digital avatar, such as, e.g., eye color, skin color, hairstyle, accessory, gender, and body type; translate the .plist file 2204 into a set of animation frames 2205 made up of 2D illustrated images, such as the frames shown in FIGS. 15-18, which may then be rendered in a timed sequence to create an animation image 2206, such as, e.g., a digital avatar with the animated facial expressions 1401, 1403, 1405, and 1407; take in a template for animation, e.g., an .swf file, and user specifications in an accept template; the template for animation may include the data and resources required for rendering a digital avatar into an animation, but may have default features, such as a standard facial image, clothing color, and/or skin color; the template for animation may then be combined with the user specifications to create a file type, e.g., a .plist, which contains both the template for animation and the user specifications in combination, wherein the user specifications may adjust the default features included in the template for animation to reflect the user selections; i.e., user can select a first animated avatar from a plurality of animated avatars 1401, 1403, 1405, and 1407 for rendering as shown in FIGS. 15, 16, 17, and 18 respectively), and
perform, in response to receiving a third input to select an object of the at least one object, an edit function of the avatar image corresponding to the selected object (O'Hara, ¶¶ [0058] and [0100]-[0102] with 1902, 1904, and 1905 in FIG. 19 and FIG. 3: after selecting the desired camera, the user may actuate a user control, such as a camera snap button 103, which may activate the selected camera, that may then be used to take a picture of either the user's face or another person's face in in an image capture step 1902; after the user takes a picture using an image capture device in an image capture step 1902, the captured image with the border 305 (i.e., emoji) are shown in FIG. 3, which may be customized in an image transformation step 1904 with further user input, during which features are applied to the captured image, e.g., smoothing, face shape, skin color, and eye color, to transform the selected or captured image to partially resemble a 2D cartoon illustration; the image and 2D illustrated body, collectively referred to herein as the digital avatar, may then open to user customization in a user customization step 1905; ¶¶ [0060] and [0067]-[0071] with FIGS. 3-4: after the image is captured, user-controls is provided for the user to adjust the size, location, and/or rotation of the image of the face with respect to the pre-determined border 101 and the eye level slot 102, so as to cause the image of the face to be centered within and fill the pre-determined border 101 and the eyes of the face to be both on the same horizontal line and centered within the eye level indicator; after the user selects or captures a face image and adjusts its position, size, and/or rotation, the user may choose a face shape 303 that may be used to generate a border 305 that crops a selected or captured face image 301 after its size, position, and rotation have been adjusted; the user can customize the border 305 around the image 301 by widening or narrowing it by dragging one or more border change buttons 302; after completing the selection and customization of a face shape, the user may actuate a user-operated control to step to the next or previous customization option, such as by tapping a forward or reverse arrow button 310; the user may tap the current customization option, such as a "Face shape" 320 label, to call up a menu of customization options and then directly go to the desired option by selecting it from the menu; ¶¶ [0087]-[0089] with FIGS.12-13: allow the user to customize the shape of the body of the avatar as shown in FIG. 12; the user may customize the shape of the body of a digital avatar 1201 underneath the image 301 by adjusting a user-operated control, such as a body size slider 1203 and/or by choosing between two gender options 1204; sliding of the body size slider 2013 may widen or narrow the body of the digital avatar 1201; the male or female gender options 1204 may change the body type of the digital avatar 1201 to reflect either a male or a female shape; the user may choose colors for different articles of clothing worn by the digital avatar 1201 by tapping the color selection button 803; allow the user to customize the color of various articles of clothing worn by the avatar as shown in FIG. 13; pressing the color selection button 803 may open a color selection wheel 901 for the user to select a color for different clothing worn by the digital avatar 1201; the user may select a color from the color selector wheel 901 and then apply the selected color to a shirt on the avatar by tapping a shirt button 1301 , to pants by tapping a pants button 1302 , and to shoes by tapping a shoes button 1303) (O'Hara, ¶¶ [0072], [0075]-[0077], [0079]-[0081], and [0083]-[0090] with FIGS. 5-9: allow the user to customize features of the avatar, such as smoothness and lighting, skin color, hairstyle, and hair color to cause the selected smoothness and lighting, skin color, hairstyle, and hair color to overlay and replace the actual smoothness and lighting, skin color, hairstyle, and hair color, as depicted in the captured image of the real face).
O'Hara further discloses a method of providing a user interface for editing an emoji while operating a camera function of an electronic device (O'Hara, ¶ [0053]-[0054]: allow the user to upload an image of their choosing or to take a picture using a camera for incorporation into the avatar; allow the user to customize the digital avatar by adding different features, e.g., glasses, hairstyle, and skin color).
O'Hara also discloses one or more non-transitory computer-readable storage media (O'Hara, ¶ [0117]: tangible memories (e.g., random access memories (RAMs), read-only memories (ROMs), and/or programmable read only memories (PROMS)), tangible storage devices (e.g., hard disk drives, CD/DVD drives, and/or flash memories))) storing one or more computer programs including computer-executable instructions (O'Hara, ¶¶ [0120]-[0121]: the software includes programming instructions may be stored on or in one or more non-transitory, tangible storage devices, such as one or more hard disk drives, CDs, DVDs, and/or flash memories; associated data may be stored in any type of volatile and/or non-volatile memory; the software may be loaded into a non-transitory memory) that, when executed by one or more processors (O'Hara, ¶ [0117]: processors) of an electronic device (O'Hara, ¶¶ [0057] and [0118]: a mobile device/computer data processing system), cause the electronic device to perform operations described above (O'Hara, ¶ [0121]: the software may be loaded into a non-transitory memory and executed by one or more processors; the programming instructions are configured to implement one or more processes and algorithms that implement one or more of the functions of the computer data processing system, as recited herein).
O'Hara fails to explicitly disclose (1) display a preview image comprising a face portion of an avatar image of a user to which a motion of a subject has been applied based on the motion of the subject obtained from the camera; and (2) receive a second input on a specific area among a plurality of areas related to the avatar image, identify, based on an intensity of the second input, a first animation to be applied into the avatar image among a plurality of animations mapped to the specific area stored in the memory.
DYE teaches a system and a method relating to user interfaces for displaying an avatars (ABSTRACT and ¶ [0002]), wherein display a preview image comprising a face portion of an avatar image of a user to which a motion of a subject has been applied based on the motion of the subject obtained from the camera (DYE, ¶¶ [0223]-[0260] with FIGS. 6C-AK: in response to detecting input 623 in FIG. 6D, device 600 activates camera 602 (e.g., switches from the rear-facing camera) and updates image display region 620 to display live camera preview 620-1 from camera 602, showing a representation of subject 632 positioned in the field-of-view of camera 602 and background 636 displayed behind subject 632 in FIG. 6E; the visual effects option affordances include avatar effects affordance 624-1 and sticker effects affordance 624-2; detects input 626 (e.g., a tap gesture on display 601) on avatar effects affordance 624-1; in response to detecting input 626 in FIG. 6F, device 600 displays avatar options menu 628 with a scrollable listing of avatar options 630; avatar options menu 628 also includes selection region 629 for indicating a selected one of avatar options 630 in FIG. 6G; as shown in FIG. 6G, robot avatar option 630-3 is positioned in selection region 629, which indicates robot avatar option 630-1 is selected; avatar options 630 correspond to a virtual avatar visual effect applied to a representation of the subject in image display region 620; specifically, each avatar option 630 corresponds to a virtual avatar that, when selected, is transposed onto the face of the subject in the image display region, while other portions of the image in image display region (such as a background or other portions of the user, such as their body) remain displayed; a user (e.g., subject 632) positioned in the field-of-view of camera 602 can control visual aspects of the virtual avatar by changing the pose (e.g., rotation or orientation) of their face, including moving various facial features (e.g., winking, sticking out their tongue, smiling, etc.); the avatar features correspond (e.g., are mapped) to one or more physical features of a user's face such that detected movement of the user's physical features (e.g., as determined based on a camera such as a depth sensing camera) affects the avatar feature (e.g., affects the feature's graphical representation); as a user's physical features (such as facial features) and position (such as head position, head rotation, or head tilt) changes, the electronic device detects the changes and modifies the displayed image of the virtual avatar to reflect the changes in the user's physical features and position; the electronic device modifies the displayed image of the virtual avatar to represent these expressions, emotions, context, tone, or other non-verbal communication; customizable avatars are virtual avatars that can be selected and customized by a user, e.g., to achieve a desired appearance (e.g., to look like the user); the customizable avatars generally have an appearance of a human character, rather than a non-human character such as an anthropomorphic construct of an animal or other nonhuman object; features of the avatar can be created or changed, if desired, using an avatar editing user interface; device 600 displays the avatar so that it remains fixed on the subject's face as he moves within the field-of-view of camera 602; device 600 also continually modifies the avatar based on any detected changes in the user's face, including pose of the face or changes in facial expressions, while maintaining the displayed subject's body and background 636 in the image shown in image display region 620; in response to detecting input 654 in FIG. 6U, device 600 displays sticker options menu 656 having a scrollable listing of stickers 658 in FIG, 6V, wherein the stickers are static graphical objects that may be selected by a user and applied to the image in image display region 620; a sticker can be selected by a touch-and-drag gesture that initiates on the selected sticker, drags the sticker to a desired location, and then places the sticker at the desired location by terminating the drag (e.g., lifting finger); in FIG. 6AE, device 600 shows the representation of the user in the image on image display region 620, but modified with visual effects that include robot avatar 633 (e.g., selected from avatar options menu 628) and helmet sticker 658-1 (e.g., selected from sticker options menu 656); in FIG. 6AJ, device 600 detects input 676 (e.g., a tap gesture on display 601) on capture affordance 621; device 600 captures an image of the live camera preview 620-1 in FIG. 6AJ, and displays, in FIG. 6AK, media item 620-2 (e.g., a still image of the state of live camera preview 620-1 at the time device 600 detected input 676 on capture affordance 621); device 600 also updates camera options region 625 to replace capture affordance 621 and camera selector affordance 627 with markup affordance 677, edit affordance 678, and send affordance 680, wherein markup affordance 677 allows a user to mark-up media item 620-2, and edit affordance 678 allows a user to edit media item 620-2 such as by cropping the image or adjusting other characteristics of media item 620-2; ¶¶ [0315]-[0342] with FIGS. 8C-8AM: in FIGS. 8E-S8, device 600 detects input 826 on avatar effects affordance 824-1, and displays avatar options menu 828 which includes a scrollable listing of avatar options 830 and selection region 829 for indicating a selected one of avatar options 830; displayed avatar options include customizable woman avatar 830-1, null avatar option 830-2, robot avatar option 830-3, rabbit avatar option 830-4, and a second customizable woman avatar 830-5; as shown in FIG. 8F, null avatar option 830-2 is selected as the default avatar option 830; in FIG. 8K, device 600 displays customizable woman avatar option 830-5 positioned within selection region 829, just prior to being selected (e.g., gesture 844b has not yet terminated), and edit affordance 804 displayed below customizable woman avatar option 830-5; device 600 displays edit affordance 804 in selection region 829 when a customizable avatar option is positioned in selection region 829; edit affordance 804 can be selected to display an avatar editing user interface; in FIG. 8L, after avatar option 830-5 is selected (e.g., gesture 844b is terminated), device 600 removes robot avatar 833 from image display region 820 (while continuing the blurred effect 803) and displays an animation of customizable woman avatar 835 appearing from the avatar option 830-5 within selection region 829 and moving to image display region 820 (e.g., at a position at the center of image display region 820, or at a location corresponding to the subject's face), where avatar 835 is modified based on detected changes to the subject's face; the animation includes the avatar (e.g., 835) starting out as a static appearance (e.g., similar to a sticker matching the displayed avatar option) and then transitioning into a dynamic state (shown in FIG. 8L) in which the avatar (e.g., 835) is modified based on device 600 detecting changes to a user's face; as shown in FIGS. 8M and 8N, when avatar 835 is applied to the subject's face, device 600 modifies avatar 835 based on detected changes to subject's face, including changes to the position of the face within the field of view of camera 602; device 600 displays different avatars on the user's head in response to detecting swipe gestures on image display region 820; e.g., in FIG. 8V, device 600 detects swipe gesture 806 moving in a rightward direction across image display region 820; in response to detecting swipe gesture 806, device 600 scrolls avatar options 830 in avatar options menu 828; as the avatar options 830 begin to scroll, the currently selected avatar option (e.g. robot avatar option 830-3 in FIG. 8V) leaves selection region 829 and, as the scroll continues, a new avatar option (e.g., avatar option 830-1 in FIG. 8W) enters selection region 829; as the avatar options scroll, device 600 also displays, in image display region 820, an animation of the currently selected avatar (e.g., robot avatar 833) moving off of the user's face and exiting the right side of image display region 820 while a next avatar (e.g., customizable avatar 831) enters image display region 820 from the left, and moves to the user's face; edit affordance 804 can be selected (e.g., via input 807 in FIG. 8Z) to display avatar editing user interface 808 shown in FIG. 8AA; avatar editing user interface 808 provides an interface for editing the avatar (e.g. customizable avatar 835) corresponding to the customizable avatar option (e.g., 830-5) associated with the selected edit affordance 804; as shown in FIG. 8AA, avatar editing user interface 808 includes a representation of customizable avatar 835 and various avatar feature options 810 (e.g., face color options 810-1 and face shape options 810-2) that represent currently selected avatar feature options and available feature options that can be selected to modify avatar 835; ¶¶ [0512]-[0517] with FIG.14D-H: device 600 modifies avatar 1421 in response to detected changes in the head and face of the subject; e.g., as shown in FIGS. 14D-14F, device 600 detects the subject's head turned to the side, while winking and smiling and, in response, modifies avatar 1421 to reflect these same movements (e.g., avatar 1421 is shown with the avatar head turned to the side with a wink and smile), while also adjusting the position of shadow 1430 based on the movement of the avatar; movement of the avatar head affects the spatial position of the avatar hair relative to representation of subject 1422; as a result, device 600 dynamically modifies some portions of the avatar based on the changed spatial relationship; e.g., as the head turns, device 600 displays some of the avatar hair moving back over the representation of the subject's shoulder and/or behind the representation of the subject's neck (e.g., device 600 hides some portions of the avatar hair), as indicated at reference number 1424-1; conversely, as the avatar head turns, device 600 also displays (e.g., reveals) portions of the avatar hair that were previously hidden prior to turning the avatar head (e.g., they were previously hidden behind the representation of the subject's neck or shoulder), as indicated at reference number 1424-2; thus, device 600 hides or displays dynamic portions of the avatar (such as portions of avatar hair) based on the change in position of the user, which correspondingly affects movement of the avatar; FIG. 14E shows representation of hand 1425 moved back towards subject 1422, close to the representation of the subject's shoulder; as representation of hand 1425 moves closer to representation of subject 1422, device 600 determines that the spatial position of representation of hand 1425 moves into the dynamic portion of avatar hair; as a result, device 600 displays additional avatar hair 1421-2 positioned in front of representation of hand 1425; similarly, FIG. 14F shows representation of hand 1425 positioned even closer towards representation of subject 1422 and a greater amount of hair displayed in front of representation of hand 1425; in FIG. 14G, the subject returns to a forward facing position with a neutral facial expression, and device 600 modifies avatar 1421 accordingly; in FIG. 14H, the subject looks up, and device 600 modifies avatar 1421 accordingly; device 600 shows avatar 1421 with a head tilting up to reveal the underside of the avatar's head, which includes blending effect 1434 at a location where the representation of the subject's neck intersects avatar 1421).
O'Hara and DYE are analogous art because they are from the same field of endeavor, a system and a method relating to user interfaces for displaying an avatars. It is well known in the art that the user can either move the camera or move the subject to adjust location of subject's image captured. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of DYE to O'Hara. Motivation for doing so would provide improved visual feedback to the user enhances the operability of the device and makes the user-device interface more efficient.
O'Hara in view of DYE fails to explicitly disclose receive a second input on a specific area among a plurality of areas related to the avatar image, identify, based on an intensity of the second input, a first animation to be applied into the avatar image among a plurality of animations mapped to the specific area stored in the memory.
Okada teaches a system and a method relating to displaying a character on a screen (Okada, Col. 1, lines 21-25), wherein receive a second input on a specific area among a plurality of areas related to the avatar image, identify, based on an duration of the second input, a first animation to be applied into the avatar image among a plurality of animations mapped to the specific area stored in the memory (Okada, Claims 1-2 in Col. 20, lines 18-51: in response to selecting one of a plurality of predefined regions of the character, the character is animated and moving the selecting member in one of a plurality of predefined movement modes relative to the touch sensitive display screen to cause the animation of the character to vary in a manner that depends directly on the selected one of the predefined regions of the character; Col. 2, lines 43-65; Col. 8, line 53 – Col. 9, line 47 and Col. 10, lines 32-48 and 65-67 with FIGS. 4 and 24A-C: determines which part of a character displayed on the display 3 is touched by a player, based on contact position data passed from the contact position information acquisition part 403; a plurality of region tables 407 (housed inside ROM 34) is prepared corresponding to image display instructions so that the performance occurring when a character is touched can be run on any of the screens; the contact mode is a concept that includes elements such as (1) the length of contact time, (2) contact frequency, (3) contact interval (i.e., interval between the time of a first contact at a first position and the time of a second contact at that first position), (4) the trajectory drawn by the contact points, and (5) the speed of movement to/between the contact points; each performance pattern table 408 (in the ROM 34) is a table containing performance pattern data coordinated to each region such as the head region, the thoracic region and the leg region and the like, and/or the contact mode; e.g., if the duration of contact to the head region is not more than 0.5 seconds, first performance pattern data (i.e., the first movement and expression of a character) is applied, if the duration of contact to the head region is not less than 0.5 seconds, second performance pattern data (i.e., the second movement and expression of a character) is applied; i.e., different animations are applied to the emoji based on the time specified between the first touch and the second touch on the same specific area; Col. 14, lines 4-36 with FIGS. 7B and 8-11: a player, using his finger F, touches the head region 701 shown in the screen displayed in FIG. 7B, between one and four times for a short time duration; after this contact has been made the screen displayed on the display 3 transitions to the screen shown in FIG. 9; i.e., displaying animation of the character touching her head region and the display of the character changes so that the character acts to show anxiety about something; if a player uses his finger F to touch the head region 701 of the screen displayed in FIG. 7B and continues this finger contact for a long duration or makes this contact for a short duration but not less than five times, displaying animation that the head part of the character 700 is touched and the change in the display of the character indicates an action of anger as shown in FIG. 10; i.e., in response to different touch inputs on the same region of emoji, display different animation at different time; Col. 14, line 41 – Col. 15, line 5 with FIGS. 12-15: a player, using his finger F, touches the thoracic region 702 shown in the screen displayed in FIG. 7B, between one and four times for a short time duration; after this contact has been made the screen displayed on the display 3 transitions to the screen shown in FIG. 13; i.e., when the character is touched in the thoracic region the display of the character changes so that the character performs actions resulting from being lightly pushed by a person; if a player uses his finger F to touch the thoracic region 702 of the screen displayed in FIG. 7B and continues this finger contact for a long duration or makes this contact for a short duration but not less than five times, displaying animation that the character acts as though pushed strongly by a person and the display of the character changes to indicate a disgruntled expression; Col. 15, lines 10-42 with FIGS.16-19: when the character is touched in the waist region the display of the character changes so that the display of the character changes to indicate an arising expression of embarrassment or the display of the character changes to indicate an expression and actions resulting from an outburst of anger by the character according to different touch input; Col. 15, line 47 – Col. 16, line 18 with FIGS.20-23: when the character is touched in the legs region the display of the character changes to indicate performance of an action to gently avoid the contact or the display of the character changes to indicate an expression and actions resulting from the character experiencing pain in her leg).
O'Hara in view DYE, and Okada are analogous art because they are from the same field of endeavor, a system and a method relating to displaying a character on a screen. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Okada to O'Hara in view DYE. Motivation for doing so would become possible to infer the intentions, mood and feelings of the user and to provide a performance that is more appealing to the user (Okada, Col. 9, lines 25-30).
O'Hara in view DYE and Okada fails to explicitly disclose identify, based on an intensity of the second input, a first animation to be applied into the avatar image.
Bernstein teaches a system and a method related to detecting inputs for manipulating user interfaces (Bernstein,¶ [0004]), wherein identify, based on an intensity of the second input, a first animation to be applied into the avatar image (Bernstein, ¶¶ [0013]-[0014]: detecting a first input that includes detecting an increase in intensity of a contact on the touch-sensitive surface; in response to detecting the first input, displaying a first animation of the user interface transitioning from the first display state to a second display state, where a rate of change from the first display state to the second display state is determined in accordance with an intensity of the contact included in the first input; after the user interface has transitioned to the second display state: detecting a second input that includes detecting an increase in intensity of a contact on the touch-sensitive surface; and in response to detecting the second input, displaying a second animation of the user interface transitioning from the second display state to a third display state; detecting a reduction in the intensity of the contact included in the second input; and in response to detecting the reduction in the intensity of the contact included in the second input, displaying a third animation of the user interface transitioning from the third display state to the second display state; ¶¶ [0183]-[0214] with FIGS. 5A1-5A18 and 5B1-5B16: in FIGS. 5A1-5A8, an example user interface 138A-02 goes through a series of changes in response to a first gesture, include displaying an animation showing a series of intermediate display states between an initial display state and an augmented display state of the user interface in accordance with changes in the intensity of the contact; during the first gesture, the intensity of the contact does not exceed a predefined intensity threshold (e.g., "ITL") prior to the end of the gesture; as a result, the user interface is returned to its initial display state after the end of the gesture is detected; in FIGS. 5A10-5A17, the user interface 138A-102 responds to an initial portion of a second gesture in largely the same manner as illustrated in FIGS. 5A1-5A8, e.g., displaying an animation showing a series of intermediate display states between the initial display state and the augmented display state in accordance with changes in the contact intensity of the initial portion of the second gesture; however, in contrast to the scenario shown in FIGS. 5A1-5A8, during the second gesture, the intensity of the contact does exceed the predefined intensity threshold (e.g., "ITL") before the end of the gesture is detected, as shown in FIG. 5A14; as a result, the user interface enters the augmented display state and remains in the augmented display state after the end of the gesture is detected; in FIGS. 5B1-5B16, an example user interface 138B-02 goes through a series of changes in response to a gesture; the changes include (1) displaying a first animation showing a transition from an initial display state to an augmented display state of the user interface in accordance with the intensity of the contact increasing to a predefined intensity threshold (e.g., a light press intensity threshold ITL) in FIG. 5B4, (2) displaying a second animation (which is a continuation of the first animation) showing a transition from the augmented display state to a terminal display state of the user interface in accordance with the intensity of the contact increasing beyond the predefined intensity threshold, and (3) displaying a third animation showing the user interface reverting from the terminal display state back to the augmented display state; user interface 138A-02 includes various example user interface elements, such as the "Tickle" button 138A-04 and various elements figuratively represented by the eyes, nose, mouth, and face of the smiley face shown in user interface 138A-02).
O'Hara in view DYE and Okada, and Bernstein are analogous art because they are from the same field of endeavor, a system and a method related to detecting inputs for manipulating user interfaces. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Bernstein to O'Hara in view DYE and Okada. Motivation for doing so would help to provide a user interface that intuitively provides users with additional information and functionality, thereby reducing the user's cognitive burden and improving the human-machine interface (Bernstein, ¶ [0032]).
Claims 3 and 12
O'Hara in view DYE, Okada, and Bernstein discloses all the elements as stated in Claims 1 and 10 respectively and further discloses wherein the specific area includes at least one of a head area, a hand area, a belly area, or a leg area of the avatar image (Okada, Col. 14, lines 4-36 with FIGS. 7B, 8-11: a player, using his finger F, touches the head region 701 shown in the screen displayed in FIG. 7B; Col. 14, line 41 – Col. 15, line 5 with FIGS. 12-15: a player, using his finger F, touches the thoracic region 702 shown in the screen displayed in FIG. 7B; Col. 15, lines 10-42 with FIGS.16-19: when the character is touched in the waist region the display of the character changes so that the display of the character changes to indicate an arising expression of embarrassment or the display of the character changes to indicate an expression and actions resulting from an outburst of anger by the character according to different touch input; Col. 15, line 47 – Col. 16, line 18 with FIGS.20-23: when the character is touched in the legs region the display of the character changes to indicate performance of an action to gently avoid the contact or the display of the character changes to indicate an expression and actions resulting from the character experiencing pain in her leg).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Okada to O'Hara in view DYE. Motivation for doing so would become possible to infer the intentions, mood and feelings of the user and to provide a performance that is more appealing to the user (Okada, Col. 9, lines 25-30).
Claims 6 and 15
O'Hara in view DYE, Okada, and Bernstein discloses all the elements as stated in Claims 1 and 10 respectively and further discloses to: stop displaying the face portion of the avatar image based on the motion of the subject in response to the first input (O'Hara, ¶¶ [0058]-[0059] with FIGS. 1 and 3: actuate a user control, such as a camera snap button 103, the location of the image of target face is no longer adjusted by moving camera or the target face) (DYE, ¶¶ [0255]-[0260] with FIG. 6AJ-AK: in FIG. 6AJ, device 600 detects input 676 (e.g., a tap gesture on display 601) on capture affordance 621; in response to detecting input 676, device 600 captures an image of the live camera preview 620-1 in FIG. 6AJ, and displays, in FIG. 6AK, media item 620-2 (e.g., a still image of the state of live camera preview 620-1 at the time device 600 detected input 676 on capture affordance 621); in FIG. 6AK, device 600 displays camera application user interface 615 showing the captured image (e.g., media item 620-2) in the image display region 620 (e.g., media item 620-2 replaces the live camera preview 620-1 shown in image display region 620); media item 620-2 is a representation of the live camera preview 620-1 at the time the capture affordance 621 was selected); and display the applied plurality of animations of the avatar image (O'Hara, ¶¶ [0091]-[0099] and [0103] with FIGS. 14-18 and 1906 in FIG. 19: a grid of animated selectable digital avatar animations may be presented, such as animated avatars 1401, 1403, 1405, and 1407; each animated avatar may present a pre-fabricated sequence of animation frames which may include layers of 2D and 3D animation and optionally text; one or more of these animation frames may be edited to include customizations dictated by the user; editing one or more drawn frames in a pre-determined set of drawn frames to reflect one or more of the customizations that the user specified; user may then signal completion of the selection by tapping a Start Now button 1409; the digital avatar may then be rendered during a render process step 1906).
Claims 7 and 16
O'Hara in view DYE, Okada, and Bernstein discloses all the elements as stated in Claims 1 and 10 respectively and further discloses wherein the at least one edit function includes a function of animating the avatar image, a function of editing the avatar image, a function of setting a background image of the avatar image, a function of generating a sticker based on the avatar image, a function of downloading an item of the avatar image, a function of downloading an animation of the avatar image, or a function of sharing the avatar image (O'Hara, ¶¶ [0058] and [0100]-[0102] with 1902, 1904, and 1905 in FIG. 19 and FIG. 3: after selecting the desired camera, the user may actuate a user control, such as a camera snap button 103, which may activate the selected camera, that may then be used to take a picture of either the user's face or another person's face in in an image capture step 1902; after the user takes a picture using an image capture device in an image capture step 1902, the captured image with the border 305 (i.e., emoji) are shown in FIG. 3, which may be customized in an image transformation step 1904 with further user input, during which features are applied to the captured image, e.g., smoothing, face shape, skin color, and eye color, to transform the selected or captured image to partially resemble a 2D cartoon illustration; the image and 2D illustrated body, collectively referred to herein as the digital avatar, may then open to user customization in a user customization step 1905; ¶¶ [0060] and [0067]-[0071] with FIGS. 3-4: after the image is captured, user-controls is provided for the user to adjust the size, location, and/or rotation of the image of the face with respect to the pre-determined border 101 and the eye level slot 102, so as to cause the image of the face to be centered within and fill the pre-determined border 101 and the eyes of the face to be both on the same horizontal line and centered within the eye level indicator; after the user selects or captures a face image and adjusts its position, size, and/or rotation, the user may choose a face shape 303 that may be used to generate a border 305 that crops a selected or captured face image 301 after its size, position, and rotation have been adjusted; the user can customize the border 305 around the image 301 by widening or narrowing it by dragging one or more border change buttons 302; after completing the selection and customization of a face shape, the user may actuate a user-operated control to step to the next or previous customization option, such as by tapping a forward or reverse arrow button 310; the user may tap the current customization option, such as a "Face shape" 320 label, to call up a menu of customization options and then directly go to the desired option by selecting it from the menu; ¶¶ [0087]-[0089] with FIGS.12-13: allow the user to customize the shape of the body of the avatar as shown in FIG. 12; the user may customize the shape of the body of a digital avatar 1201 underneath the image 301 by adjusting a user-operated control, such as a body size slider 1203 and/or by choosing between two gender options 1204; sliding of the body size slider 2013 may widen or narrow the body of the digital avatar 1201; the male or female gender options 1204 may change the body type of the digital avatar 1201 to reflect either a male or a female shape; the user may choose colors for different articles of clothing worn by the digital avatar 1201 by tapping the color selection button 803; allow the user to customize the color of various articles of clothing worn by the avatar as shown in FIG. 13; pressing the color selection button 803 may open a color selection wheel 901 for the user to select a color for different clothing worn by the digital avatar 1201; the user may select a color from the color selector wheel 901 and then apply the selected color to a shirt on the avatar by tapping a shirt button 1301 , to pants by tapping a pants button 1302 , and to shoes by tapping a shoes button 1303; ¶¶ [0072], [0075]-[0077], [0079]-[0081], and [0083]-[0090] with FIGS. 5-9: allow the user to customize features of the avatar, such as smoothness and lighting, skin color, hairstyle, and hair color to cause the selected smoothness and lighting, skin color, hairstyle, and hair color to overlay and replace the actual smoothness and lighting, skin color, hairstyle, and hair color, as depicted in the captured image of the real face) (O'Hara, ¶¶ [0091]-[0099], [0103], and [0115]-[0117] with FIGS. 14-18, 1906 in FIG. 19, and FIGS. 22-23: various animated avatar previews may be created and presented based on the customization selections made by the user; a grid of animated selectable digital avatar animations may be presented, such as animated avatars 1401, 1403, 1405, and 1407; each animated avatar may present a pre-fabricated sequence of animation frames which may include layers of 2D and 3D animation and optionally text; one or more of these animation frames may be edited to include customizations dictated by the user; editing one or more drawn frames in a pre-determined set of drawn frames to reflect one or more of the customizations that the user specified; the user may select one of the customized animations, such as by tapping the animation; the user may then signal completion of the selection by tapping a Start Now button 1409; the digital avatar may then be rendered during a render process step 1906; create a .plist file 2204, which may include the specifications for the digital avatar, such as, e.g., eye color, skin color, hairstyle, accessory, gender, and body type; translate the .plist file 2204 into a set of animation frames 2205 made up of 2D illustrated images, such as the frames shown in FIGS. 15-18, which may then be rendered in a timed sequence to create an animation image 2206, such as, e.g., a digital avatar with the animated facial expressions 1401, 1403, 1405, and 1407; take in a template for animation, e.g., an .swf file, and user specifications in an accept template; the template for animation may include the data and resources required for rendering a digital avatar into an animation, but may have default features, such as a standard facial image, clothing color, and/or skin color; the template for animation may then be combined with the user specifications to create a file type, e.g., a .plist, which contains both the template for animation and the user specifications in combination, wherein the user specifications may adjust the default features included in the template for animation to reflect the user selections; i.e., user can select a first animated avatar from a plurality of animated avatars 1401, 1403, 1405, and 1407 for rendering as shown in FIGS. 15, 16, 17, and 18 respectively) (DYE, ¶¶ [0315]-[0342] with FIGS. 8C-8AM: in FIGS. 8E-8F, device 600 detects input 826 on avatar effects affordance 824-1, and displays avatar options menu 828 which includes a scrollable listing of avatar options 830 and selection region 829 for indicating a selected one of avatar options 830; displayed avatar options include customizable woman avatar 830-1, null avatar option 830-2, robot avatar option 830-3, rabbit avatar option 830-4, and a second customizable woman avatar 830-5; as shown in FIG. 8F, null avatar option 830-2 is selected as the default avatar option 830; in FIG. 8K, device 600 displays customizable woman avatar option 830-5 positioned within selection region 829, just prior to being selected (e.g., gesture 844b has not yet terminated), and edit affordance 804 displayed below customizable woman avatar option 830-5; device 600 displays edit affordance 804 in selection region 829 when a customizable avatar option is positioned in selection region 829; edit affordance 804 can be selected to display an avatar editing user interface; in FIG. 8L, after avatar option 830-5 is selected (e.g., gesture 844b is terminated), device 600 removes robot avatar 833 from image display region 820 (while continuing the blurred effect 803) and displays an animation of customizable woman avatar 835 appearing from the avatar option 830-5 within selection region 829 and moving to image display region 820 (e.g., at a position at the center of image display region 820, or at a location corresponding to the subject's face), where avatar 835 is modified based on detected changes to the subject's face; the animation includes the avatar (e.g., 835) starting out as a static appearance (e.g., similar to a sticker matching the displayed avatar option) and then transitioning into a dynamic state (shown in FIG. 8L) in which the avatar (e.g., 835) is modified based on device 600 detecting changes to a user's face; as shown in FIGS. 8M and 8N, when avatar 835 is applied to the subject's face, device 600 modifies avatar 835 based on detected changes to subject's face, including changes to the position of the face within the field of view of camera 602; device 600 displays different avatars on the user's head in response to detecting swipe gestures on image display region 820; e.g., in FIG. 8V, device 600 detects swipe gesture 806 moving in a rightward direction across image display region 820; in response to detecting swipe gesture 806, device 600 scrolls avatar options 830 in avatar options menu 828; as the avatar options 830 begin to scroll, the currently selected avatar option (e.g. robot avatar option 830-3 in FIG. 8V) leaves selection region 829 and, as the scroll continues, a new avatar option (e.g., avatar option 830-1 in FIG. 8W) enters selection region 829; as the avatar options scroll, device 600 also displays, in image display region 820, an animation of the currently selected avatar (e.g., robot avatar 833) moving off of the user's face and exiting the right side of image display region 820 while a next avatar (e.g., customizable avatar 831) enters image display region 820 from the left, and moves to the user's face; edit affordance 804 can be selected (e.g., via input 807 in FIG. 8Z) to display avatar editing user interface 808 shown in FIG. 8AA; avatar editing user interface 808 provides an interface for editing the avatar (e.g. customizable avatar 835) corresponding to the customizable avatar option (e.g., 830-5) associated with the selected edit affordance 804; as shown in FIG. 8AA, avatar editing user interface 808 includes a representation of customizable avatar 835 and various avatar feature options 810 (e.g., face color options 810-1 and face shape options 810-2) that represent currently selected avatar feature options and available feature options that can be selected to modify avatar 835; in response to detecting selection of sticker effects affordance 824-2, device 600 displays sticker options menu 856 having a scrollable listing of sticker 858 in FIG. 8AL; the stickers are static graphical objects that may be selected by a user and applied to the image in image display region 820; a sticker can be selected by a tap gesture, and the corresponding sticker is then displayed at a position on the image display region 820; a sticker can be selected by a touch-and-drag gesture that initiates on the selected sticker, drags the sticker to a desired location, and then places the sticker at the desired location by terminating the drag (e.g., lifting finger); in FIG. 8AL, device 600 detects a selection of capture affordance 821, which causes device 600 to capture an image of the live camera preview 820-1 displayed in image display region 820, including the visual effects ( e.g., customized avatar 835 and helmet sticker 858-1), as well as other image data including subject 832 and background 836; in FIG. 8AM, device 600 displays camera application user interface 815 showing the captured image (e.g., media item 820-2) in the image display region 820 (e.g., media item 820-2 replaces the live camera preview 820-1 shown in image display region 820); media item 820-2 is a representation of the live camera preview 820-1 at the time the capture affordance 821 was selected; device 600 also updates camera options region 825 to replace capture affordance 821 and camera selector affordance 827 with markup affordance 877, edit affordance 878, and share affordance 880; Markup affordance 877 allows a user to mark-up media item 820-2; edit affordance 878 allows a user to edit media item 820-2 such as by cropping the image or adjusting other characteristics of media item 820-2; share affordance 880 allows a user to send media item 820-2 to another device, such as, e.g. in a messaging application or email application).
Claims 2, 4, 8-9, 11, 13, and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over O'Hara in view DYE, Okada, and Bernstein as applied to Claims 1, 7, 10, and 16, respectively above, and further in view of Marks et al. (US 2007/0260984 A1, pub. date: 11/08/2007), hereinafter Marks.
Claims 2 and 11
O'Hara in view DYE, Okada, and Bernstein discloses all the elements as stated in Claims 1 and 10 respectively and further discloses a microphone (O'Hara, ¶ [0117]: microphone) (DYE, ¶¶ [0067], [0074]-[0075], [0125]-[0126], [0147], [0171], and [0206] with 113 in FIGS. 1A, 2, and 4A: microphone) (Bernstein, ¶ [0042], [0050]-[0051], [0094]-[0095], [0116], and [0140] with 113 in FIGS. 1A, 2, and 4A: microphone),
O'Hara in view DYE, Okada, and Bernstein fails to explicitly disclose to receive, through the microphone, a user utterance related to at least one of an expression or an operation of the avatar image.
Marks teaches a system and a method relating to animating an avatar (Marks, ABSTRACT), wherein receive, through a microphone, a user utterance related to at least one of an expression or an operation of the avatar image (Marks, ¶¶ [0039]-[0040] with FIGS. 2D-K: if the ambient microphones 102B detect music in the background environment an avatar might automatically start dancing in beat with the rhythm of the real world music; another avatar would automatically become agitated by the same music and speech balloons filled with "turn down the music" would appear over the avatar's head; beyond automated response from the avatar based on a user-selected theme the users could control the avatar in real time using voice commands inputted through a microphone or using the controller; if a user wanted their avatar to have a specific facial expression commands issued through the controller or special voice commands would initiate a particular facial expression).
O'Hara in view DYE and Marks are analogous art because they are from the same field of endeavor, a system and a method relating to animating an avatar. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Marks to O'Hara in view DYE. Motivation for doing so would improve and enhance the communication experience (.
Claims 4 and 13
O'Hara in view DYE, Okada, Bernstein, and Marks discloses all the elements as stated in Claims 2 and 11 respectively and further discloses to: apply a second animation related to the at least one of the expression or the operation to the avatar image based on the user utterance; and display the applied second animation of the avatar image (Marks, ¶¶ [0039]-[0040] with FIGS. 2D-K: if the ambient microphones 102B detect music in the background environment an avatar might automatically start dancing in beat with the rhythm of the real world music; another avatar would automatically become agitated by the same music and speech balloons filled with "turn down the music" would appear over the avatar's head; beyond automated response from the avatar based on a user-selected theme the users could control the avatar in real time using voice commands inputted through a microphone or using the controller; if a user wanted their avatar to have a specific facial expression commands issued through the controller or special voice commands would initiate a particular facial expression).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Marks to O'Hara in view DYE. Motivation for doing so would improve and enhance the communication experience (.
Claims 8 and 17
O'Hara in view DYE, Okada, and Bernstein discloses all the elements as stated in Claims 7 and 16 respectively and further discloses wireless communication circuitry (O'Hara, ¶ [0119]: the computer data processing system may include one or more computers at the same or different locations; when at different locations, the computers may be configured to communicate with one another through a wired and/or wireless network communication system) (DYE, ¶¶ [0073] and [0094] with 108 in FIG. 1A; Bernstein, ¶¶ [0049] and [0065] with 108 in FIG. 1A: RF (radio frequency) circuitry 108 receives and sends RF signals, also called electromagnetic signals; RF circuitry 108 converts electrical signals to/from electromagnetic signals and communicates with communications networks and other communications devices via the electromagnetic signals; RF circuitry 108 optionally includes well-known circuitry for performing these functions, including but not limited to an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a CODEC chipset, a subscriber identity module (SIM) card, memory, and so forth),
wherein (O'Hara, ¶¶ [0091]-[0099], [0103], and [0115]-[0117] with FIGS. 14-18, 1906 in FIG. 19, and FIGS. 22-23: various animated avatar previews may be created and presented based on the customization selections made by the user; a grid of animated selectable digital avatar animations may be presented, such as animated avatars 1401, 1403, 1405, and 1407; each animated avatar may present a pre-fabricated sequence of animation frames which may include layers of 2D and 3D animation and optionally text; one or more of these animation frames may be edited to include customizations dictated by the user; editing one or more drawn frames in a pre-determined set of drawn frames to reflect one or more of the customizations that the user specified; the user may select one of the customized animations, such as by tapping the animation; the user may then signal completion of the selection by tapping a Start Now button 1409; the digital avatar may then be rendered during a render process step 1906; create a .plist file 2204, which may include the specifications for the digital avatar, such as, e.g., eye color, skin color, hairstyle, accessory, gender, and body type; translate the .plist file 2204 into a set of animation frames 2205 made up of 2D illustrated images, such as the frames shown in FIGS. 15-18, which may then be rendered in a timed sequence to create an animation image 2206, such as, e.g., a digital avatar with the animated facial expressions 1401, 1403, 1405, and 1407; take in a template for animation, e.g., an .swf file, and user specifications in an accept template; the template for animation may include the data and resources required for rendering a digital avatar into an animation, but may have default features, such as a standard facial image, clothing color, and/or skin color; the template for animation may then be combined with the user specifications to create a file type, e.g., a .plist, which contains both the template for animation and the user specifications in combination, wherein the user specifications may adjust the default features included in the template for animation to reflect the user selections; i.e., user can select a first animated avatar from a plurality of animated avatars 1401, 1403, 1405, and 1407 for rendering as shown in FIGS. 15, 16, 17, and 18 respectively).
O'Hara in view DYE, Okada, and Bernstein fails to explicitly disclose download, from an external electronic device, a second animation of the avatar image through the wireless communication circuitry based on a fourth input for selecting an object corresponding to the function of the downloading the animation of the avatar image among the at least one object, and update the applied first animation of the avatar image with the downloaded second animation.
Marks teaches a system and a method relating to animating an avatar (Marks, ABSTRACT), wherein download, from an external electronic device, a second animation of the avatar image through the wireless communication circuitry based on a fourth input for selecting an object corresponding to the function of the downloading the animation of the avatar image among the at least one object, and update the applied first animation of the avatar image with the downloaded second animation (Marks, ¶¶ [0032], [0040]-[0045] and [0065] with FIGS. 1 and 3-4: users could also manually control specific aspect of the avatar, and initiate Real-Time Effects (RTE) that would be imposed on, or incorporated into, their avatar; FIG. 3 shows example groups of the predefined video effects library 108A; the pre-defined video effects 108A are video effects that may be loaded onto the computer system 100 by the system manufacturer or downloaded over the network 128; FIG. 4 shows the partial contents of the custom video effects library 108B; a user can create custom video effects by creating or editing avatar animations to their liking; a user would be able to visit a website or custom video effect portal where they would be able to download custom video effects after paying a license or sales fee over the network 128; i.e., a user can download custom video effects for applying to avatar animations in response to a fourth touch input, e.g., visit a website or custom video effect portal to select a video effect from FIG. 3 or FIG. 4, and pre-defined video effects or custom video effects selected and downloaded from portal would be imposed on, or incorporated into, their avatar to replace previous animation)
O'Hara in view DYE, Okada, and Bernstein, and Marks are analogous art because they are from the same field of endeavor, a system and a method relating to animating an avatar. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Marks to O'Hara in view DYE, Okada, and Bernstein. Motivation for doing so would provide more customization options for generating animation.
Claims 9 and 18
O'Hara in view DYE, Okada, Bernstein, and Marks discloses all the elements as stated in Claims 8 and 17 respectively and further discloses to: store, in the memory, the second animation downloaded from the external electronic device (Marks, ¶¶ [0032], [0040]-[0045] and [0065] with FIGS. 1 and 3-4: users could also manually control specific aspect of the avatar, and initiate Real-Time Effects (RTE) that would be imposed on, or incorporated into, their avatar; FIG. 3 shows example groups of the predefined video effects library 108A; the pre-defined video effects 108A are video effects that may be loaded onto the computer system 100 by the system manufacturer or downloaded over the network 128; FIG. 4 shows the partial contents of the custom video effects library 108B; a user can create custom video effects by creating or editing avatar animations to their liking; a user would be able to visit a website or custom video effect portal where they would be able to download custom video effects after paying a license or sales fee over the network 128; i.e., a user can download video effects and store in video effects library 108A shown in FIG. 3 for applying to avatar animations).
O'Hara in view DYE, Okada, and Bernstein, and Marks are analogous art because they are from the same field of endeavor, a system and a method relating to animating an avatar. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Marks to O'Hara in view DYE, Okada, and Bernstein. Motivation for doing so would provide more customization options for generating animation.
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over O'Hara in view DYE, Okada, and Bernstein as applied to Claims 1 and 10 respectively above, and further in view of Takechi et al. (US 2019/0138266 A1, filed on 09/23/2016), hereinafter Takechi.
Claims 5 and 14
O'Hara in view DYE, Okada, and Bernstein discloses all the elements as stated in Claims 1 and 10 respectively and except failing to explicitly disclose further discloses to: identify a third animation related to a designated operation designated to induce the second input or the third input based on the second input or the third input not being received for a designated time after displaying the user interface; and apply the third animation to the avatar image and display the applied third animation of the avatar image.
Takechi teaches a system and a method relating to displaying a character image (Takechi, ¶ [0001]), wherein identify a third animation related to a designated operation designated to induce the second input or the third input based on the second input or the third input not being received for a designated time after displaying the user interface; and applying the third animation to the avatar image and displaying the applied third animation of the avatar image (Takechi, ¶ [0107]: the display section has a function of displaying a character, wherein the character is a person, an animal or the like appearing in an animation; ¶¶ [0208]-[0210] with S206-S208 in FIG. 18; FIG. 1 and 18: when the user's face is not detected by the analysis section or the touch button 26 is not touched a predetermined time before the wake-up time, the state becomes the summoned state, and when the user's face is not detected by the analysis section or the touch button 26 is not touched until the wake-up time, display an image to wake the user up, and the output a sound to wake the user up, and starts the wake-up function; i.e., display animation of a character image to wake the user up until detecting the user's face or the touch button 26 is touched when the user's face is not detected or the touch button 26 is not touched for a predetermined time since the state becomes the summoned state).
and Takechi are analogous art because they are from the same field of endeavor, a system and a method relating to animating an avatar. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention to apply the teaching of Takechi to O'Hara in view DYE, Okada, and Bernstein. Motivation for doing so would provide action or alert that keep the user interested and informed, and thus enhance user experience .
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
RICKWALD et al. (US 2018/0336715 A1, pub. date: 11/22/2018) discloses in ¶¶ [0301]-[0451] with FIGS. 10A-10I, 11A-11C, 12A-12C, 13, 14A-14D, 15A-15B, 16A-16B, and 17A-17B that (1) the virtual avatar includes an avatar face having one or more avatar features (e.g., avatar facial features); (2) the avatar features correspond (e.g., are mapped) to one or more physical features of a user's face such that detected movement of the user's physical feature affect the avatar feature (e.g., affect the feature's graphical representation); (3) as a user's physical features (such as facial features) and position (such as head position or head tilt) changes, the electronic device detects the changes and modifies the displayed image of the virtual avatar to reflect the changes in the user's physical features and position; (4) the changes to the user's physical features and position are indicative of various expressions, emotions, context, tone, or other non-verbal communication; (5) the electronic device modifies the displayed image of the virtual avatar to represent these expressions, emotions, context, tone, or other non-verbal communication; (6) the magnitude of a reaction of an avatar feature (e.g., a discrete element of the avatar that can be moved or modified discretely with respect to other avatar features) corresponds to a magnitude of a change in a physical feature of the user (e.g., a detected or tracked feature such as a user muscle, muscle group, or anatomical feature such as an eye); e.g., the magnitude of the change in the physical feature is, in some embodiments, determined in accordance with a potential range of motion of the physical feature, wherein the magnitude is representative of a relative position of the physical feature within the range of motion (e.g., the predicted or modeled range of motion) of that physical feature; (7) the magnitude of the reaction of the avatar feature is similarly a relative position of the avatar feature within a range of motion of the avatar feature; (8) the magnitude of change is determined based on a comparison or measurement (e.g., a distance) of the starting position and ending position of the physical feature, through the change; (8) the change in the physical feature may be translated to a modification of the first avatar feature by applying the measured change in the physical feature to the avatar feature (e.g., directly or as a scaled or adjusted value); (8) modifications to an avatar feature have both a magnitude component and a directional component, with the directional component of the modification in the avatar feature being based on a directional component of a change in one or more of the physical features that the avatar feature is reactive to; (9) in some embodiments, the direction of a reaction of an avatar feature corresponds (e.g., directly or inversely) to a relative direction of a change in a physical feature of the user, wherein the relative direction of the change in the physical feature is determined based on a direction of movement of the physical feature from an initial position (e.g., a neutral, resting position of the physical feature or, in some embodiments, a position of the physical feature that is initially detected by the device); (10) in some embodiments, the direction of the reaction of the avatar feature corresponds directly (e.g., the physical feature moves up, the avatar feature also moves up) to the relative direction of the change in the physical feature; (11) in other embodiments, the direction of the reaction of the avatar feature corresponds inversely (e.g., the physical feature moves up, the avatar feature moves down) to the relative direction of the change in the physical feature; (12) the directional component of the change in the avatar feature is mirrored with respect to the directional component of the change in the physical feature; e.g., when the physical feature (e.g., user's mouth) moves left, the avatar feature (e.g., avatar mouth) moves right; (13) in some embodiments, the directional component of the change in the avatar feature is the same as the directional component of the change in the physical feature for movement along a vertical axis and mirrored for movement along a horizontal axis, similar to the effect seen when looking in a mirror; (14) in some embodiments, the neutral, resting position of a user's iris is determined to be a particular position (e.g., centered) relative to the perimeter of the user's eyeball; (15) the electronic device detects the user transitioning between various positions of the user's physical features, and then updates or modifies the virtual avatar (e.g., poo, bear, alien, rabbit, robot, unicorn, chicken, or pig avatar) in accordance with the detected changes in the user's physical features; and (16) in these examples, the electronic device transitions display of the virtual avatar between various expressions and positions by maintaining characteristics of certain features of the virtual avatar and animating changes to other virtual avatar features; e.g., as shown in 1001A-1001D of FIG. 10A, the device maintains the position of the avatar eyes 1030 while modifying other avatar features, such as the avatar mouth 1030, eyebrows 1038, and upper portion 1034. RICKWALD further discloses in ¶¶ [0667]-[0669] with FIG. 26E-H that (1) device 600 displays an image display region 2630 presenting a live camera preview 2630-1 from camera 602 showing a representation of subject 2632 positioned in the field-of-view of camera 602 and background 2636 displayed behind subject 2632; (2) virtual avatar 2633 can be displayed on the representation of subject 2632 (e.g., on the subject's face); (3) a user (e.g., subject 2632) positioned in the field-of-view of camera 602 can control visual aspects of the virtual avatar by changing the pose (e.g., rotation or orientation) of their face, including moving various facial features (e.g., winking, sticking out their tongue, smiling, etc.); e.g., FIG. 26G shows the virtual avatar 2633 tilting its head in response to device 600 detecting movement of the user's head in the field of view of camera 602; (4) similarly, FIG. 26H shows device 600 modifying the virtual avatar's facial features (e.g., mouth 2634) in response to detecting a change in the physical features of the user's face (e.g., the user's mouth).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HWEI-MIN LU whose telephone number is (313)446-4913. The examiner can normally be reached Mon - Fri: 9:00 AM - 6:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela D. Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HWEI-MIN LU/Primary Examiner, Art Unit 2142
Prosecution Insights