DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement
3. The Information Disclosure Statement filed 22 December 2025 has been fully considered by Examiner. An annotated copy is included herewith.
Claim Rejections - 35 USC § 103
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
5. Claims 1-3, 6, 7, 9-12, 15, 16, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Axen (US-2010/0050083) in view of Jetley (US-2017/0308770).
Regarding claim 1: Axen discloses a method comprising: receiving a design document (fig 1(14) and [0064] of Axen – structured (HTML) document data defining design for rendering); generating a rendered image of the design document (fig 1(12-20) and [0065]-[0066] of Axen); determining one or more animation parameters (fig 2, [0074]-[0077], [0084], and [0140] of Axen – HTML/VSML web content determines animation content based on content parsing and/or feature selection); and generating an animated output by at least applying the one or more animation parameters to the design document, the generating of the animated output transforms the design document into an animated design document (fig 4, [0071], and [0139]-[0140] of Axen – generating output animation design for rendering based on design document animation parameters and user selections).
Axen does not disclose generating, via a machine learning model, a saliency mask by providing a representation of the rendered image as input to the machine learning model, the saliency mask indicates one or more regions of visual importance within the rendered image; and based at least in part on the saliency mask, generating the animated output.
Jetley discloses generating, via a machine learning model, a saliency mask by providing a representation of the rendered image as input to the machine learning model, the saliency mask indicates one or more regions of visual importance within the rendered image (fig 2(S104-S108) and [0040]-[0043] of Jetley – neural network generates saliency mask/map to generate a cropped image including salient regions); and based at least in part on the saliency mask, generating the output (fig 2 (S112-S114) and [0042]-[0044] of Jetley).
Axen and Jetley are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to generate, via a machine learning model, a saliency mask by providing a representation of the rendered image as input to the machine learning model, the saliency mask indicating one or more regions of visual importance within the rendered image, and, based at least in part on the saliency mask, generate the output, as taught by Jetley. By combination with Axen, the output of Jetley would be animated output. The motivation for doing so would have been to more efficiently process the image/video data based on portions that seem most relevant, utilizing the more efficient means of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen according to the relied-upon teachings of Jetley to obtain the invention as specified in claim 1.
Regarding claim 2: Axen in view of Jetley discloses the method of claim 1 (as rejected above), wherein the one or more animation parameters include at least one of, one or more characteristics associated with the animated output or one or more animation rules that dictate how the animation output should be applied to different elements within a design document ([0070]-[0073], and [0139]-[0140] of Axen – characteristics for animation and other features to be rendered according to the design document).
Regarding claim 3: Axen in view of Jetley discloses the method of claim 1 (as rejected above), wherein the generation of the animation output is further based on a single user input representative of a request to convert the design document into the animation output ([0140] of Axen).
Regarding claim 6: Axen in view of Jetley discloses the method of claim 1 (as rejected above). Axen does not disclose converting the saliency mask into a binary image using a threshold value; and combine elements of a scene graph that are within a threshold distance to each other into one or more clusters based on using the binary image, and wherein the generating of the animation output is further based on the converting and the combining.
Jetley discloses converting the saliency mask into a binary image using a threshold value ([0058]-[0059] of Jetley – converted based on threshold for eye-fixation); and combine elements of a scene graph that are within a threshold distance to each other into one or more clusters based on using the binary image ([0087]-[0088] of Jetley – grouped within a bounding box according to saliency), and wherein the generating of the animation output is further based on the converting and the combining (fig 2(S112) and [0043]-[0044] of Jetley).
Axen and Jetley are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to convert the mask into a binary image using a threshold value, and combine elements of a scene graph that are within a threshold distance to each other into one or more clusters based on using the binary image, and wherein the generating of the sequence is further based on the converting and the combining, as taught by Jetley. By combination with Axen, the sequence of Jetley would be an animation sequence. The motivation for doing so would have been to more efficiently process the image/video data based on portions that seem most relevant, utilizing the more efficient means of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen according to the relied-upon teachings of Jetley to obtain the invention as specified in claim 6.
Regarding claim 7: Axen in view of Jetley discloses the method of claim 1 (as rejected above). Axen does not disclose wherein the generating of the saliency mask is based on training the machine learning model on a dataset of images with labeled regions of visual importance.
Jetley discloses wherein the generating of the saliency mask is based on training the machine learning model on a dataset of images with labeled regions of visual importance (fig 2(S102-S104), fig 3(S202,S204,71), [0039]-[0040], and [0058]-[0059] of Axen – based on labeled regions of eye fixation coordinates).
Axen and Jetley are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to generate the saliency mask based on training the machine learning model on a dataset of images with labeled regions of visual importance, as taught by Jetley. The motivation for doing so would have been to more efficiently process the image/video data based on portions that seem most relevant, utilizing the more efficient means of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen according to the relied-upon teachings of Jetley to obtain the invention as specified in claim 7.
Regarding claim 9: Axen in view of Jetley discloses the method of claim 1 (as rejected above), further comprising: determining animation presets that include a set of predefined animation styles that are selectable by a user ([0071]-[0073] of Axen), and wherein the determining of the one or more animation parameters include determining how the animation presets are applied or changed, and wherein the generating of the animation output is based on the determining how the animation presets are applied or changed ([0086]-[0087], and [0131]-[0133] of Axen – end user can change animation parameters for generating and outputting the animation).
Regarding claim 10: Axen discloses a system (fig 1, fig 2, fig 4, [0064], and [0097] of Axen) comprising: a memory component (fig 1(12) and [0064] of Axen – data read into system, so some form of memory component is implicit); and a processing device coupled to the memory component (fig 2(24) and [0078] of Axen – processes received data, so processor 24 is coupled to the memory component), the processing device to perform operations ([0026] and [0078] of Axen) comprising: receiving an image or file that includes one or more elements (fig 1(14), [0058], and [0064] of Axen); determining one or more animation rules (fig 2, [0074]-[0077], [0084], and [0140] of Axen – HTML/VSML web content determines animation content based on content parsing and/or feature selection); and generating an animation sequence of the one or more elements of the image or file by at least applying the one or more animation rules (fig 4, [0071], and [0139]-[0140] of Axen – generating animation based on design document animation parameters and user selections).
Axen does not disclose generating a mask that indicates one or more regions of visual importance in the image or file; based at least in part on the mask, determining one or more animation rules.
Jetley discloses generating a mask that indicates one or more regions of visual importance in the image or file (fig 2(S104-S108) and [0040]-[0043] of Jetley –generates saliency mask/map to indicate salient/important regions of image for cropping and rendering); and based at least in part on the mask, determining one or more rules (fig 2 (S112-S114) and [0042]-[0044] of Jetley – determine rules for new image processing based on saliency).
Axen and Jetley are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to generate a mask that indicates one or more regions of visual importance in the image or file, and, based at least in part on the mask, determine one or more rules, as taught by Jetley. By combination with Axen, the rules of Jetley would be animation rules. The motivation for doing so would have been to more efficiently process the image/video data based on portions that seem most relevant, utilizing the more efficient means of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen according to the relied-upon teachings of Jetley to obtain the invention as specified in claim 10.
Regarding claim 11: Axen in view of Jetley discloses the system of claim 10 (as rejected above). Axen does not disclose wherein the mask is a saliency mask, and wherein the automatic generation of the mask includes automatically generating, via a saliency model, the saliency mask, and wherein the saliency mask is a greyscale image or a heat map that indicates which pixels of the one or more regions are likely to attract human attention.
Jetley discloses wherein the mask is a saliency mask (fig 2(S108) and [0042]-[0043] of Jetley – saliency mask/map to generate a cropped image including salient regions), and wherein the automatic generation of the mask includes automatically generating, via a saliency model, the saliency mask fig 2(S104-S108) and [0040]-[0043] of Jetley – neural network generates saliency mask/map to generate a cropped image including salient regions, and wherein the saliency mask is a greyscale image or a heat map that indicates which pixels of the one or more regions are likely to attract human attention ([0051] of Jetley – saliency mask/map is greyscale image indicating eye gaze fixation).
Axen and Jetley are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to have the mask be a saliency mask, and wherein the automatic generation of the mask includes automatically generating, via a saliency model, the saliency mask, and wherein the saliency mask is a greyscale image or a heat map that indicates which pixels of the one or more regions are likely to attract human attention, as taught by Jetley. The motivation for doing so would have been to more efficiently process the image/video data based on portions that seem most relevant, utilizing the more efficient means of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen according to the relied-upon teachings of Jetley to obtain the invention as specified in claim 11.
Regarding claim 12: Axen in view of Jetley discloses the system of claim 10 (as rejected above), wherein the generation of the animation sequence is further based on a single user input representative of a request to convert the image or file into the animation sequence ([0054]-[0057], [0073], and [0133] of Axen).
Regarding claim 15: Axen in view of Jetley discloses the system of claim 10 (as rejected above). Axen does not disclose wherein the operations further comprising: converting the mask into a binary image using a threshold value; and combine elements of a scene graph that are within a threshold distance to each other into one or more clusters based on using the binary image, and wherein the generating of the animation sequence is further based on the converting and the combining.
Jetley discloses wherein the operations further comprising: converting the mask into a binary image using a threshold value ([0058]-[0059] of Jetley – converted based on threshold for eye-fixation); and combine elements of a scene graph that are within a threshold distance to each other into one or more clusters based on using the binary image ([0087]-[0088] of Jetley – grouped within a bounding box according to saliency), and wherein the generating of the sequence is further based on the converting and the combining (fig 2(S112) and [0043]-[0044] of Jetley).
Axen and Jetley are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to convert the mask into a binary image using a threshold value, and combine elements of a scene graph that are within a threshold distance to each other into one or more clusters based on using the binary image, and wherein the generating of the sequence is further based on the converting and the combining, as taught by Jetley. By combination with Axen, the sequence of Jetley would be an animation sequence. The motivation for doing so would have been to more efficiently process the image/video data based on portions that seem most relevant, utilizing the more efficient means of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen according to the relied-upon teachings of Jetley to obtain the invention as specified in claim 15.
Regarding claim 16: Axen in view of Jetley discloses the system of claim 10 (as rejected above). Axen does not disclose wherein the generating of the mask is based on providing a representation of the image to a machine learning model as input and training the machine learning model on a dataset of images with labeled regions of visual importance.
Jetley discloses wherein the generating of the mask is based on providing a representation of the image to a machine learning model as input and training the machine learning model on a dataset of images with labeled regions of visual importance (fig 2(S102-S104), fig 3(S202,S204,71), [0039]-[0040], and [0058]-[0059] of Jetley – based on labeled regions of eye fixation coordinates).
Axen and Jetley are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to generate the mask based on providing a representation of the image to a machine learning model as input and training the machine learning model on a dataset of images with labeled regions of visual importance, as taught by Jetley. The motivation for doing so would have been to more efficiently process the image/video data based on portions that seem most relevant, utilizing the more efficient means of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen according to the relied-upon teachings of Jetley to obtain the invention as specified in claim 16.
Regarding claim 18: Axen in view of Jetley discloses the system of claim 10 (as rejected above), wherein the operations further comprising: determining animation presets that include a set of predefined animation styles that are selectable by a user ([0071]-[0073] of Axen), and wherein the determining of the one or more animation rules include determining how the animation presets are applied or changed, and wherein the generating of the animation sequence is based on the determining how the animation presets are applied or changed ([0086]-[0087], and [0131]-[0133] of Axen – end user can change animation parameters for generating and outputting the animation).
Regarding claim 19: Axen discloses a non-transitory computer-readable medium (fig 1(12) and [0064] of Axen – data read into system, so some form of physical memory component is implicit) storing executable instructions, which when executed by a processing device, cause the processing device to perform operations (fig 2(24), [0026], and [0078] of Axen – processes received data, so processor 24 is coupled to the memory component) comprising: the image including one or more design elements (fig 1(12-20) and [0065]-[0066] of Axen); filtering a representation of the one or more design elements of the image based on predetermined criteria (fig 2, [0074]-[0077], [0084], and [0140] of Axen – filtered according to HTML/VSML web content); detecting one or more hero elements from the filtered representation of the one or more design elements based at least in part on the filtering ([0070], [0080]-[0081], [0088] of Axen – detected key elements and biographical information based on parsing of the data structures); and generating an animated output associated with the image based at least in part on the detecting of the one or more hero elements (fig 4, [0071], and [0139]-[0140] of Axen – generating output animation design for rendering based on design document animation parameters and user selections, including detected essential (hero) elements).
Axen does not disclose generating, via a machine learning model, a mask that indicates one or more portions of an image that are likely to attract human attention; and detecting one or more hero elements from the filtered representation of the one or more design elements based at least in part on the mask, the one or more hero elements indicate one or more regions of visual importance.
Jetley discloses generating, via a machine learning model, a mask that indicates one or more portions of an image that are likely to attract human attention (fig 2(S104-S108) and [0040]-[0043] of Jetley – neural network generates saliency mask/map to generate a cropped image including salient regions, which will also be regions likely to attract human attention); and detecting one or more hero elements from the filtered representation of the one or more design elements based at least in part on the mask, the one or more hero elements indicate one or more regions of visual importance (fig 2(S108-S110) and [0042]-[0044] of Jetley).
Axen and Jetley are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to generate, via a machine learning model, a mask that indicates one or more portions of an image that are likely to attract human attention, and detect one or more hero elements from the filtered representation of the one or more design elements based at least in part on the mask, the one or more hero elements indicating one or more regions of visual importance, as taught by Jetley. The motivation for doing so would have been to more efficiently process the image/video data based on portions that seem most relevant, utilizing the more efficient means of machine learning. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen according to the relied-upon teachings of Jetley to obtain the invention as specified in claim 19.
6. Claims 4, 5, 13 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Axen (US-2010/0050083) in view of Jetley (US-2017/0308770), and in further view of Herman (US-2016/0078662).
Regarding claim 4: Axen in view of Jetley discloses the method of claim 1 (as rejected above). Axen in view of Jetley does not disclose generating, from the design document, a scene graph that represents each element in the design document in a hierarchical structure where each node in the scene graph corresponds to an element or group of elements in the design document.
Herman discloses generating, from the design document, a scene graph that represents each element in the design document in a hierarchical structure where each node in the scene graph corresponds to an element or group of elements in the design document (fig 4, [0037]-[0038], and [0208]-[0209] of Herman).
Axen and Herman are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to generate, from the design document, a scene graph that represents each element in the design document in a hierarchical structure where each node in the scene graph corresponds to an element or group of elements in the design document, as taught by Herman. The motivation for doing so would have been to more efficiently organize video data for processing and rendering. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen further according to the relied-upon teachings of Herman to obtain the invention as specified in claim 4.
Regarding claim 5: Axen in view of Jetley, and in further view of Herman, discloses the method of claim 4 (as rejected above), Axen in view of Jetley does not disclose wherein the generation of the animation output is further based on filtering the scene graph by selecting or discarding specific elements from the scene graph based on predefined criteria.
Herman discloses wherein the generation of the animation output is further based on filtering the scene graph by selecting or discarding specific elements from the scene graph based on predefined criteria (fig 11 and [0301] of Herman – motion path for animation manipulated by, among other things, creating and deleting notes and edges from the scene graph based on animation criteria).
Axen and Herman are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to further base the generation of the animation output on filtering the scene graph by selecting or discarding specific elements from the scene graph based on predefined criteria, as taught by Herman. The motivation for doing so would have been to more efficiently organize video data for processing and rendering. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen further according to the relied-upon teachings of Herman to obtain the invention as specified in claim 5.
Regarding claim 13: Axen in view of Jetley discloses the system of claim 10 (as rejected above), wherein the image is representative of a rendered document image, the rendered document image being a visual representation of a design document, the design document being the file created in graphic design or layout software ([0058], and [0064]-[0065] of Axen).
Axen in view of Jetley does not disclose wherein the operations further comprising: generating, from the design document, a scene graph that represents each element in the design document in a hierarchical structure where each node in the scene graph corresponds to an element or group of elements in the design document.
Herman discloses wherein the operations further comprising: generating, from the design document, a scene graph that represents each element in the design document in a hierarchical structure where each node in the scene graph corresponds to an element or group of elements in the design document (fig 4, [0037]-[0038], and [0208]-[0209] of Herman).
Axen and Herman are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to generate, from the design document, a scene graph that represents each element in the design document in a hierarchical structure where each node in the scene graph corresponds to an element or group of elements in the design document, as taught by Herman. The motivation for doing so would have been to more efficiently organize video data for processing and rendering. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen further according to the relied-upon teachings of Herman to obtain the invention as specified in claim 13.
Regarding claim 14: Axen in view of Jetley, and in further view of Herman, discloses the system of claim 13 (as rejected above). Axen in view of Jetley does not disclose wherein the generation of the animation sequence of the one or more elements is further based on filtering the scene graph by selecting or discarding specific elements from the scene graph based on predefined criteria.
Herman discloses wherein the generation of the animation sequence of the one or more elements is further based on filtering the scene graph by selecting or discarding specific elements from the scene graph based on predefined criteria (fig 11 and [0301] of Herman – motion path for animation manipulated by, among other things, creating and deleting notes and edges from the scene graph based on animation criteria).
Axen and Herman are analogous art because they are from the same field of endeavor, namely image and video data processing. Before the effective filing date of the invention, it would have been obvious to one of ordinary skill in the art to further base the generation of the animation output on filtering the scene graph by selecting or discarding specific elements from the scene graph based on predefined criteria, as taught by Herman. The motivation for doing so would have been to more efficiently organize video data for processing and rendering. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Axen further according to the relied-upon teachings of Herman to obtain the invention as specified in claim 14.
Allowable Subject Matter
7. Claims 8, 17 and 20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James A Thompson whose telephone number is (571)272-7441. The examiner can normally be reached M-F 8am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alicia Harrington can be reached at 571-272-2330. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JAMES A THOMPSON/Primary Examiner, Art Unit 2615
Prosecution Insights