CTNF 19/007,346 CTNF 87216 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia 1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. Priority 02-26 AIA 2. Receipt is acknowledged of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file. Information Disclosure Statement 06-52 3. The information disclosure statement (IDS) submitted on 06/02/2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner. Claim Rejections - 35 USC § 103 07-06 AIA 15-10-15 4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-20-aia AIA 5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA 6. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA 7. Claim (s) 1, 8-11 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Le Tuan (US 2008/0049025 A1) in view of Yang et al. (US 2020/0151508 A1) . 8. With reference to claim 1, Le Tuan teaches A computer-implemented method, (“The present invention provides a method and an apparatus for producing computer-generated animation which allows an original visual or audio object in an animation to be divided into a plurality of sub-objects to be individually edited such that the sub-objects inherit characteristics and attributes of the original object.” [0019]) Le Tuan also teaches extracting visual elements of a static infographic, the static infographic being an image described in a graphic design file; (“A graphical object is often a natural combination of elementary objects which are themselves compound objects. For example, a piece of text is composed of sentences, which are themselves composed of words, which are themselves composed of individual letters. Other graphical images, such as bit maps, call also be arbitrarily decomposed into individual fragments of any shape, such as the pieces of a jigsaw puzzle, a tessellation or vectorial objects, which can themselves be decomposed into sub-fragments.” [0053] “One embodiment of the present invention includes typographic attributes, such as: font size, font name, leading, tracking, Adobe multiple master and TrueType GX font variations and attributes, as well as classic attributes, such as bold, italic, underline, etc. Dynamic array 420 can be expanded to accommodate additional visual attributes as required. … in FIG. 4 element 402 includes an attribute called color which is associated with evolution in time 440. Evolution in time 440 provides a linkage between a particular time and a particular value. In one embodiment, evolution in time 440 is implemented as a function which determines how a color of a visual object evolves over time. Every other attribute within element 402 is similarly associated with an evolution in time, although some attributes may have evolutions that remain static over time.” [0070-0071] “Browser-Based Text Animation Design The present invention may be used to enable users to create text animation via web browsers. The architecture of the web based text animation system is illustrated in FIG. 8. A web browser 800 is accessed by a user. The web browser accesses a web server 804 via the Internet 802. The web server 804 is coupled to a text animation engine 808 that enables the user to create the text animation.” [0078-0079]) Le Tuan further teaches recommending a dynamic effect and applying the dynamic effect to the visual elements based on the structure of the static infographic to generate an animated infographic. (“Dynamic array 420 can be expanded to accommodate additional visual attributes as required. For animation purposes, each attribute within an element is associated with an evolution in time, which is a member of an animation class. For example, in FIG. 4 element 402 includes an attribute called color which is associated with evolution in time 440. Evolution in time 440 provides a linkage between a particular time and a particular value. In one embodiment, evolution in time 440 is implemented as a function which determines how a color of a visual object evolves over time. Every other attribute within element 402 is similarly associated with an evolution in time, although some attributes may have evolutions that remain static over time. FIG. 5 is a diagram illustrating the appearance of a graphical user interface display 500 for use in creating and editing an animation in accordance with an aspect of tie present invention.” [0070-0072] “At step 706, the system animates sub-objects separately. For example, the sub-objects )nay proceed along different parts. The system next advances to step 708. At step 708, the system recombines the sub-objects into the original object. The system next advances to step 710 which is an end state. The above-mentioned sequence of operations produces an interesting visual effect in which an object can be divided into sub-objects which fly apart and follow their own parts to eventually recombine into the original object.” [0077]) PNG media_image1.png 792 292 media_image1.png Greyscale Le Tuan does not explicitly teach determining, based on the visual elements, a structure of the static infographic indicating a layout of the visual elements in the static infographic; training a machine learning model based on datasets of the visual elements; effect based on the machine learning model; This is what Yang teaches (“The layout generation system 116 then generates a refined digital image layout 120 that is visually pleasing by determining an arrangement of the graphic elements in relation to each other based on the semantic and geometric parameters using machine learning. An example 132 of a refined digital image layout 120 is illustrated in which the vector graphic 130 and the digital image 126 are arranged next to each other horizontally and above the text block 128 to form a visually pleasing layout, which is performed automatically and without user intervention by the layout generation system 116.” [0039] “FIG. 2 depicts a system 200 in an example implementation showing operation of the generative adversarial network system 122 of FIG. 1 in greater detail as performing machine-learning training for digital image layout refinement. FIG. 3 depicts a system 300 in an example implementation showing operation of a generator module of FIG. 2 to generate a refined training digital image layout from a training digital image layout using machine learning. FIG. 4 depicts a system 400 in an example implementation showing operation of a wireframe rendering discriminator module of FIG. 2 to generate wireframe prediction data based on the refined training digital image layout using machine learning to train the generator module. FIG. 5 depicts an example implementation showing examples of wireframe rendering of graphic elements. FIG. 6 depicts a procedure 600 in an example implementation in which a wireframe digital image layout is generated by rasterizing a refined training digital image layout received from a generator module by a wireframe rendering discriminator module to train the generator module using machine learning as part of a GAN system.” [0043] “FIG. 2 describe an example 200 of an overall architecture of the generative adversarial network system 122 as part of the layout generation system 116. The layout generation system 116 includes an input generation module 202 that is configured to generate training digital image layouts 204 automatically and without user intervention that includes a plurality of graphic elements 206. The graphic elements 206, for instance, may be selected from a storage system based on a variety of parameters, including semantic and geometric parameters, which are then incorporated into a single training digital image layout 204 that is to be refined by a generator module and trained using a discriminator module as part of a generative adversarial network system 122.” [0045]) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang into Le Tuan, in order to refine digital image layouts in a manner that is visually pleasing. 9. With reference to claim 8, Le Tuan teaches applying the dynamic effect comprises at least one of: specifying an animation sequence of the visual elements, the animation sequence indicating a temporal sequence for displaying the visual elements; specifying staging of the visual elements, the staging indicating how to show the visual elements in a hierarchical way; and applying an animation effect to the visual elements. (“Dynamic array 420 can be expanded to accommodate additional visual attributes as required.” [0070] “At step 602, the object is decomposed into a plurality of sub-objects. The system next advances to step 604. At step 604, sub-objects are represented as a position of an anchor point, and a relative position of the sub-object. The system next advances to step 606. At step 606, the system determines the position of the anchor point at time T. The system next advances to step 608. At step 608, the system determines a relative position of a sub-object relative to the anchor point at time T. The system next advances to step 610. At step 610, the system combines the position of the anchor point, and the relative position of the sub-object at time T. For example, in the case of a system which uses matrix operations, the combination process includes multiplying a matrix representing an anchor point with a matrix representing a position of the sub-object relative to the anchor point. The system next advances to step 612. At step 612, the sub-objects are recombined to construct a graphical representation of the original object at time T. The system next advances to step 614. At step 614, the system outputs the graphical representation to a display. The system next advances to step 616 which is an end state. This process is iteratively repeated to create a sequence of graphical images comprising an animation. … The method for representing a sub-object as a position of an anchor point and a position of the sub-object relative to the anchor point can be generalized for other attributes besides position. For example, consider the attribute of color. An object may be decomposed into sub-objects, each of which inherit a baseline color intensity value representing a level of ambient lighting, which changes as a function of time. FIG. 7 is a flowchart illustrating the sequence of operations involved in creating an animation effect in accordance with an aspect of the present invention. … At step 706, the system animates sub-objects separately. For example, the sub-objects )nay proceed along different parts. The system next advances to step 708. At step 708, the system recombines the sub-objects into the original object. The system next advances to step 710 which is an end state. The above-mentioned sequence of operations produces an interesting visual effect in which an object can be divided into sub-objects which fly apart and follow their own parts to eventually recombine into the original object.” [0075-0077]) 10. With reference to claim 9, Le Tuan teaches the animation sequence is determined by at least one of: determining an animation sequence of the visual elements based on a reading order; and determining an animation sequence of the visual elements based on sematic tags of the visual elements. (“An Animated Text Sequence starts with a collection of glyphs, letters, words, or other text components, appearing individually or collectively in a certain order and graphically rendered with certain effects, with such effects evolving over time in any fashion, and resulting in a final frame that is similar to a static annotation where a subset of, or the full content of, the original text sequence can be read.” [0014] “At step 602, the object is decomposed into a plurality of sub-objects. The system next advances to step 604. At step 604, sub-objects are represented as a position of an anchor point, and a relative position of the sub-object. The system next advances to step 606. At step 606, the system determines the position of the anchor point at time T. The system next advances to step 608. At step 608, the system determines a relative position of a sub-object relative to the anchor point at time T. The system next advances to step 610. At step 610, the system combines the position of the anchor point, and the relative position of the sub-object at time T. For example, in the case of a system which uses matrix operations, the combination process includes multiplying a matrix representing an anchor point with a matrix representing a position of the sub-object relative to the anchor point. The system next advances to step 612. At step 612, the sub-objects are recombined to construct a graphical representation of the original object at time T. The system next advances to step 614. At step 614, the system outputs the graphical representation to a display. The system next advances to step 616 which is an end state. This process is iteratively repeated to create a sequence of graphical images comprising an animation.” [0075] “FIG. 7 is a flowchart illustrating the sequence of operations involved in creating an animation effect in accordance with an aspect of the present invention. … At step 706, the system animates sub-objects separately. For example, the sub-objects )nay proceed along different parts. The system next advances to step 708. At step 708, the system recombines the sub-objects into the original object. The system next advances to step 710 which is an end state. The above-mentioned sequence of operations produces an interesting visual effect in which an object can be divided into sub-objects which fly apart and follow their own parts to eventually recombine into the original object.” [0075-0077]) 11. With reference to claim 10, Le Tuan teaches the dynamic effects as an output, wherein the output comprises a fading animation effect, an appearing animation effect, a zooming animation effect, a wiping animation effect, or a flying in and out animation effect, and recommends one or more dynamic effect options for each visual element within a unit. (“Dynamic array of elements 410 contains a plurality of audio and visual elements involved in the animation. In this example, dynamic array of elements 410 includes elements 400, 401 and 402. These elements can be stored to and subsequently retrieved from a persistent store such as an object- oriented database management system. In the embodiment illustrated in FIG. 4, an element is a member of a C++ class, which propagates attributes through inheritance to other elements derived from the element. In FIG. 4, element 401 is an audio element, which includes a dynamic array 420 which is associated with the audio element. As is shown in FIG. 4, dynamic array 420 includes attributes such as volume, stereo pan, chorus, flanger, vibrato, tremelo, high filter, band reject, echo arid phaser. Other embodiments include other commonly known audio attributes. Element 402 is a visual element, which includes a dynamic array 430 for visual element 402.” [0059] “Dynamic array 420 can be expanded to accommodate additional visual attributes as required. For animation purposes, each attribute within an element is associated with an evolution in time, which is a member of an animation class. For example, in FIG. 4 element 402 includes an attribute called color which is associated with evolution in time 440. Evolution in time 440 provides a linkage between a particular time and a particular value. In one embodiment, evolution in time 440 is implemented as a function which determines how a color of a visual object evolves over time. Every other attribute within element 402 is similarly associated with an evolution in time, although some attributes may have evolutions that remain static over time. FIG. 5 is a diagram illustrating the appearance of a graphical user interface display 500 for use in creating and editing an animation in accordance with an aspect of tie present invention.” [0070-0072] “FIG. 6 is a flowchart illustrating the sequence of steps involved in decomposing and recombining an object to create animation effects in accordance with an aspect of the present invention. …the sub-objects are recombined to construct a graphical representation of the original object at time T. The system next advances to step 614. At step 614, the system outputs the graphical representation to a display.” [0075] “FIG. 7 is a flowchart illustrating the sequence of operations involved in creating an animation effect in accordance with an aspect of the present invention. … The above-mentioned sequence of operations produces an interesting visual effect in which an object can be divided into sub-objects which fly apart and follow their own parts to eventually recombine into the original object.” [0077]) Le Tuan does not explicitly teach the machine learning model comprises a neural network that takes property as an input, wherein the input comprises a width, a height, a shape, or a layout of each element, and wherein the machine learning model for each visual element. This is what Yang teaches (“The generator module, once trained as part of the GAN system, is then configured to refine digital image layouts without the wireframe rendering discriminator module. The generator module, for instance, may receive an input digital image layout having graphic elements that are associated with semantic parameters (e.g., tags identifying a type of graphic element such as text, image, and so forth) and geometric parameters, e.g., shape, size, and so on. The digital image layout, for instance, may be specified as a collection of graphic elements without a set arrangement by a user.” [0028] “A “generator module” is configured to generate candidates, e.g., a refined training digital image layout from a training digital image layout using a neural network.” [0031]) Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Yang into Le Tuan, in order to refine digital image layouts in a manner that is visually pleasing. 12. Claim 11 is similar in scope to claim 1, and thus is rejected under similar rationale. Le Tuan additionally teaches A device, comprising: a processing unit; and a memory coupled to the processing unit and including instructions stored thereon, the instructions, when executed by the processing unit, causing the device to perform acts (“a device capable of communicating with a server, a server coupled to a text animation engine, such that the text animation engine includes an object-oriented data structure for representing the text, and each character in the text is represented by an object in the object-oriented data structure. In embodiments of the invention, the device is a wireless device.” [0032] “FIG. 1 includes display 100, keyboard 110, mouse 120, disk 140 and CPU 170, which are connected together through bus 150, which is an I/O bus. CPU 170 additionally connects to memory 130 through bus 180, which is a faster processor-to-memory bus. Display t(o) is a display device, such as a computer monitor, for displaying the computer animation. Keyboard 110 and mouse 120 are input devices for accepting user input into the computer system. Memory 130 and disk 140 are storage devices for storing programs mid data for producing computer-generated animations. CPU 170 is a central processing unit which: accepts user input from keyboard 110 and mouse 120; processes programs and data stored within memory 130 and disk 140; and produces output for display 100. Memory 130 contains object data structures 160, which are data structures corresponding to audio and visual objects associated with an animation.” [0052]) 13. Claim 18 is similar in scope to the combination of claims 8 and 9, and thus is rejected under similar rationale. 14. Claim 19 is similar in scope to claim 10, and thus is rejected under similar rationale. 15. Claim 20 is similar in scope to claim 1, and thus is rejected under similar rationale. Le Tuan additionally teaches A computer program product stored in a computer storage medium and including computer-executable instructions, the computer-executable instructions, when executed by a device, causing the device to perform acts (“a device capable of communicating with a server, a server coupled to a text animation engine, such that the text animation engine includes an object-oriented data structure for representing the text, and each character in the text is represented by an object in the object-oriented data structure. In embodiments of the invention, the device is a wireless device.” [0032] “FIG. 1 includes display 100, keyboard 110, mouse 120, disk 140 and CPU 170, which are connected together through bus 150, which is an I/O bus. CPU 170 additionally connects to memory 130 through bus 180, which is a faster processor-to-memory bus. Display t(o) is a display device, such as a computer monitor, for displaying the computer animation. Keyboard 110 and mouse 120 are input devices for accepting user input into the computer system. Memory 130 and disk 140 are storage devices for storing programs mid data for producing computer-generated animations. CPU 170 is a central processing unit which: accepts user input from keyboard 110 and mouse 120; processes programs and data stored within memory 130 and disk 140; and produces output for display 100. Memory 130 contains object data structures 160, which are data structures corresponding to audio and visual objects associated with an animation.” [0052]) Allowable Subject Matter 12-151-08 AIA 07-43 12-51-08 16. Claim s 2-7 and 12-17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is an examiner’s statement of reasons for allowance: Regarding claims 2 and 12, the prior art of record fails to either individually or in combination teach the claimed feature of “determining a number of repeating units based on a frequency of repetition of the visual elements; constructing repeating units based on the determined number of repeating units; and determining the layout of the visual elements in the static infographic based on the constructed repeating units.” Claims 3-7 also objected to for depending from claim 2. Claims 13-17 also objected to for depending from claim 12. Conclusion 17. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michelle Chin whose telephone number is (571)270-3697. The examiner can normally be reached on Monday-Friday 8:00 AM-4:30 PM. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http:/Awww.uspto.gov/interviewpractice. If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Kent Chang can be reached on (571)272-7667. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https:/Awww.uspto.gov/patents/apply/patent- center for more information about Patent Center and https:/Awww.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. /MICHELLE CHIN/ Primary Examiner, Art Unit 2614 Application/Control Number: 19/007,346 Page 2 Art Unit: 2614 Application/Control Number: 19/007,346 Page 3 Art Unit: 2614 Application/Control Number: 19/007,346 Page 4 Art Unit: 2614 Application/Control Number: 19/007,346 Page 5 Art Unit: 2614 Application/Control Number: 19/007,346 Page 6 Art Unit: 2614 Application/Control Number: 19/007,346 Page 7 Art Unit: 2614 Application/Control Number: 19/007,346 Page 8 Art Unit: 2614 Application/Control Number: 19/007,346 Page 9 Art Unit: 2614 Application/Control Number: 19/007,346 Page 10 Art Unit: 2614 Application/Control Number: 19/007,346 Page 11 Art Unit: 2614 Application/Control Number: 19/007,346 Page 12 Art Unit: 2614 Application/Control Number: 19/007,346 Page 13 Art Unit: 2614 Application/Control Number: 19/007,346 Page 14 Art Unit: 2614 Application/Control Number: 19/007,346 Page 15 Art Unit: 2614 Application/Control Number: 19/007,346 Page 16 Art Unit: 2614
Prosecution Insights