Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Objections
Claims 5-17 objected to under 37 CFR 1.75(c) as being in improper form because a multiple dependent claim. See MPEP § 608.01(n). Accordingly, the claims are not been further treated on the merits.
Unless noted otherwise, such as for issues with antecedent basis, claims 5-17 are interpreted as depending from the first claim in the range.
Claim Interpretation
The scope of the pending claims is interpreted as follows:
Claims 13 are interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Claims 1-12, 14-17 are given their broadest reasonable interpretation.
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Such claim limitation(s) is/are:
Unit
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 16 rejected under 35 U.S.C. § 101 because the claimed invention is directed to non-statutory subject matter.
The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because the broadest reasonable interpretation of the claim is inclusive of computer programs. Computer programs on their own lack any tangible, structural form (see MPEP 2106.03). The current claim(s) do not recite sufficient structural limitations to encompass only patent eligible subject matter.
Additional structural limitations such as a non-transitory computer readable medium storing a computer program or other computer readable media defined in such a way as to exclude signal transmission media would recite subject matter within one of the four statutory categories of invention.
Claims 17 rejected under 35 U.S.C. 101 because the broadest reasonable interpretation (BRI) of computer program product is inclusive of signal media and other forms of transmission media.
Machine readable media can encompass forms of signal transmission media that falls outside of the four statutory categories of invention. MPEP 2106; citing In re Nuijten, 500 F.3d 1346, 84 USPQ2d 1495 (Fed. Cir. 2007). A claim whose BRI covers both statutory and non-statutory embodiments embraces subject matter that is not eligible for patent protection and therefore is directed to non-statutory subject matter. MPEP 2106.
Applicant’s specification does not explicitly define the claimed medium as excluding transitory forms of signal transmission. The broadest reasonable interpretation of the claimed medium in view of Applicant’s specification would therefore include ineligible subject matter. Additional structural limitations such as a non-transitory computer readable medium storing a computer program or other computer readable media defined in such a way as to exclude signal transmission media would recite subject matter within one of the four statutory categories of invention.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Use of indicates a limitation is not explicitly disclosed by the reference alone.
Claim(s) 1-3, 13-17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Takada (US 2018/0137632) in view of Zheng (US 2021/0183165)
Claim 1
Takada discloses a method for drawing an effect image, comprising:
PNG
media_image1.png
335
342
media_image1.png
Greyscale
in response to an effect drawing request triggered by a user, performing trajectory tracking on a target video provided by the user to obtain a three-dimensional point set corresponding to a motion trajectory of a target object in the target video (Takada, ¶ 35, 40: “configured to detect objects features (i.e., reference points or features or corners) over a specified region in the reference frame….The object pattern tracking module 122 is configured to estimate motion of the detected feature between the two frames (i.e., the reference frame and each current frame). For example, the object pattern tracking module 122 performs a RANSAC-type sampling step to find an initial estimate of the motion from the sampled corners/features and then refines the estimated motion by optimizing a robust probabilistic motion model of sparse corners.;
performing a map processing on the target (Takada, ¶ 43: “provided to use the tracking based information received from the hybrid tracking system 120 to perform one or more processes (e.g., digital video effects, image editing or the like)”); and
displaying, in the target video, the target effect image corresponding to the motion trajectory of the target object (Takada, ¶ 43: “manipulate the video data before it is presented on display device 140 as a visual-based application, which shows a modified tracked object in the video in one exemplary aspect”)
Takada does not explicitly disclose, but Zheng discloses performing a meshing processing on the three-dimensional point set to obtain a target mesh structure (¶ 164; Fig. 12: “Although FIG. 12 shows five white dots are used to adjust five vertices of the cube, an actual operation is not limited thereto. For example, four, six, or seven vertices may be used, or even eight or more white dots may alternatively be used to respectively adjust eight vertices of the cube.”);
performing a map processing on the target mesh structure (Zheng, ¶ 180: “a three-dimensional mapping unit, which may be configured to project the target three-dimensional model onto each of the second frame of image to the m.sup.th frame of image of the to-be-processed video based on the poses of the camera coordinate systems of the second frame of image to the m.sup.th frame of image of the to-be-processed video relative to the world coordinate system”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider meshing as claimed.
One of ordinary skill in the art would have motivation to combine in order to map the relationship between the effect object and tracked features in the scene. One of ordinary skill in the art would have had a reasonable expectation of success because Takada considers tracking a sequences of vertices in order to apply effects.
Claim 2
Takada discloses 2. The method of claim 1, wherein the performing trajectory tracking on a target video provided by the user to obtain a three-dimensional target point set corresponding to a motion trajectory of a target object in the target video comprises:
recognizing the target object in the target video (Takada, ¶ 35: “Moreover, the objects or features to be identified in the frame or specified region can be user-specified (e.g., by a user of the editing software), identified automatically based on the editing software or the like.”);
sampling the motion trajectory of the target object in the target video to obtain at least one target sampling point (Takada ¶ 4: “object tracking refers to interactive motion analysis in which a user-specified feature (i.e., an image patch) is tracked over time and the estimated motion (e.g., affine or projective) is then applied to some target image or visual effect so that it “match moves” to the tracked feature; and
Takada does not disclose, but Zheng discloses mapping the at least one target sampling point to a three-dimensional space coordinate system to obtain the three-dimensional point set (Zheng, ¶ 180: “a three-dimensional mapping unit, which may be configured to project the target three-dimensional model onto each of the second frame of image to the m.sup.th frame of image of the to-be-processed video based on the poses of the camera coordinate systems of the second frame of image to the m.sup.th frame of image of the to-be-processed video relative to the world coordinate system”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider meshing as claimed.
One of ordinary skill in the art would have motivation to combine in order to map the relationship between the effect object and tracked features in the scene. One of ordinary skill in the art would have had a reasonable expectation of success because Takada considers tracking a sequences of vertices in order to apply effects.
Claim 3
Takada discloses 3. The method of claim 2, wherein the sampling the motion trajectory of the target object in the target video to obtain at least one target sampling point comprises:
sampling the motion trajectory of the target object from the target video to obtain a two-dimensional point set, the two-dimensional point set comprising a plurality of first sampling points (Takada, ¶ 41: “sampling step is iterated as many times as the probability that the object pattern tracking module 122 determines at least one outlier-free”);
resampling the two-dimensional point set to obtain a plurality of second sampling points (Takada, ¶ 41: “sampling step is iterated as many times as the probability that the object pattern tracking module 122 determines at least one outlier-free”); and
determining, from the plurality of second sampling points, at least one target sampling point meeting a trajectory smoothing condition (Takada, ¶ 41: “during processing of the video data, the sampling step is iterated as many times as the probability that the object pattern tracking module 122 determines at least one outlier-free sample becomes sufficiently close to one. Finally, the model that best fits the detected correspondences in terms of the model evidence is selected among the sample”).
Claim 13
Examiner’s Interpretation:
Unit is interpreted under 35 U.S.C. 112, sixth paragraph as covering the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
Claim Mapping:
The same teachings and rationales in claims 1-12 are appliable to claim 14, with Takada disclosing the corresponding apparatus (Fig. 1:
PNG
media_image2.png
343
424
media_image2.png
Greyscale
Claim 14
The same teachings and rationales in claims 1-12 are appliable to claim 14, with Takada disclosing an electronic device, comprising: a processor, a memory, and an output device; the memory storing computer executable instructions; the processor executing the computer executable instructions stored in the memory, to cause the processor to be configured with the method (Fig. 1:
PNG
media_image2.png
343
424
media_image2.png
Greyscale
Claim 15
Examiner’s Interpretation:
Machine readable media can encompass forms of signal transmission media that falls outside of the four statutory categories of invention. MPEP 2106; citing In re Nuijten, 500 F.3d 1346, 84 USPQ2d 1495 (Fed. Cir. 2007). A claim whose BRI covers both statutory and non-statutory embodiments embraces subject matter that is not eligible for patent protection and therefore is directed to non-statutory subject matter. MPEP 2106.
Applicant’s specification defines computer-readable storage medium as:
a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, a computer readable signal medium may include a data signal propagated in baseband or as part of a carrier, where the computer readable program code is carried. Such propagated data signals may take a variety of forms including, but not limited to, electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium that may send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.
(Specification, ¶ 292)(Explicitly distinguishing storage medium from signal medium)
Claim 15 as drafted recites computer-readable storage medium.
Because of the explicit definition, the broadest reasonable interpretation of the claimed medium in view of Applicant’s specification covers only eligible subject matter.
Claim Mapping:
The same teachings and rationales in claims 1-12 are applicable to claim 15.
Claim 16
The same teachings and rationales in claims 1-12 are applicable to claim 16.
Claim 17
The same teachings and rationales in claims 1-12 are applicable to claim 17.
Claim(s) 7, 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Takada (US 2018/0137632) in view of Zheng (US 2021/0183165) and Wu (US 2025/0191254)
Claim 7
Takada does not explicitly disclose, but Zheng discloses 7. The method of any of claims 1 to 6, wherein the performing a meshing processing on the three- dimensional point set to obtain a target mesh structure comprises:
triangulating the whole area contained in the three-dimensional point set to obtain a triangular mesh surface (e.g. cube Mesh Fig. 12); and
determining the triangular mesh surface as a target mesh structure corresponding to the surface effect type (¶ 7: “a plurality of feature points of a model surface of the target three-dimensional model falling on the target plane”); and
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider meshing as claimed.
One of ordinary skill in the art would have motivation to combine in order to map the relationship between the effect object and tracked features in the scene. One of ordinary skill in the art would have had a reasonable expectation of success because Takada considers tracking a sequences of vertices in order to apply effects.
Takada as modified by Zheng does not disclose, but Wu discloses:
wherein the performing a map processing on the target mesh structure to obtain a target effect image comprises:
performing a texture mapping on the three-dimensional mesh surface to obtain a three-dimensional rendered target texture image (Wu, ¶ 33; “a plurality of vertices with a wider coverage area can be obtained, so that more vertices are utilized to generate a texture curve, and the texture curve is utilized to generate a more accurate effect line frame, thereby achieving continuous and smooth display of effect lines, and improving the display accuracy and effect of the effect lines.”); and
performing an edge smoothing processing on the three-dimensional rendered target texture image to obtain the target effect image (Wu, ¶ 33: “a plurality of vertices with a wider coverage area can be obtained, so that more vertices are utilized to generate a texture curve, and the texture curve is utilized to generate a more accurate effect line frame, thereby achieving continuous and smooth display of effect lines, and improving the display accuracy and effect of the effect lines.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider smoothing and textures as claimed.
One of ordinary skill in the art would have motivation to combine in order to map the relationship between the effect object and tracked features in the scene. One of ordinary skill in the art would have had a reasonable expectation of success because Takada considers tracking a sequences of vertices in order to apply effects.
Claim 10
Takada as modified by Zheng does not disclose, but Wu discloses 10. The method of any of claims 1 to 6, wherein the performing a meshing processing on the three- dimensional point set to obtain a target mesh structure comprises:
performing interpolation on pairwise adjacent sampling points in the three-dimensional point set to obtain a target point set (Wu, ¶ 35-36: “vertex expansion on the collected feature points to obtain more vertices. By means of the more vertices, a line expansion can be realized, and by means of the line expansion…The plurality of vertices are utilized to generate texture curves corresponding to adjacent pairs of vertices to obtain at least one curve.”);
generating a strip mesh on the contour region corresponding to the target point set to obtain the target mesh structure (Wu, ¶ 3, 44: “ A relatively common technology is to dynamically construct a suitable strip model through the human body feature points, and obtain effect lines after the width and curve of a strip are set smoothly…when the target object refers to the human face or facial features, the step used to collect a plurality of contour points formed by an object contour of a target object in an image to be collected may comprise: recognizing, based on a human body key boundary recognition algorithm, the plurality of contour points forming the object contour of the target object from the image to be collected. A skeleton positioning & tracking algorithm may also be used to recognize the plurality of contour points forming the object contour of the target object from the image to be collected.”);
performing a map processing on the target mesh structure to obtain the target effect image comprises:
performing trajectory rendering on the target mesh structure to obtain a three-dimensional strip trajectory map (¶ 52: “Map the effect line frame onto the image to be processed to obtain a target image corresponding to the image to be processed.”); and
determining the target effect image based on the three-dimensional strip trajectory map (¶ 52: “Map the effect line frame onto the image to be processed to obtain a target image corresponding to the image to be processed.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider contouring as claimed.
One of ordinary skill in the art would have motivation to combine in order to map the relationship between the effect object and tracked features in the scene. One of ordinary skill in the art would have had a reasonable expectation of success because Takada considers tracking a sequences of vertices in order to apply effects.
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Takada (US 2018/0137632) in view of Zheng (US 2021/0183165) and Chepizhenko (US Patent 11,024,101)
Claim 12
Takada as modified by Zheng does not disclose but Chepizhenko discloses 12. The method of any of claims 1 to 11, further comprising:
determining a trigger time of the effect drawing request triggered by the user (“Such a collection may be made available for a specified time period, such as the duration of an event to which the content relates;
determining a target holiday type corresponding to the trigger time according to a time period corresponding to at least one holiday type (“he display of LENS features 820 can be organized based on categories, such as industry, event (e.g., holiday) animals, etc. selected by a user. A 2D overlay can then be added (1006), which can be autocropped and/or centered (1008). Music/sound can then be added (1010) via an uploaded sound file, e.g., from the media tray 816. A trigger from triggers 818 can then be added (1012). A post-trigger action can then be defined (1014) for the trigger, e.g., activating the selected LENS feature(s) or a portion of the selected LENS feature. A LENS name and/or icon can then be added (1016) and the completed LENS is saved (1018) to the media library 814.”); and
obtaining a texture image pre-associated with the target holiday type, wherein the performing a map processing on the target mesh structure to obtain a target effect image comprises: performing the map processing on the target mesh structure by using the texture image to obtain the target effect image (“For example, some embodiments may involve generating a three dimensional mesh model of the object or objects, and using transformations and animated textures of the model within the video to achieve the transformation. In other embodiments, tracking of points on an object may be used to place an image or texture (which may be two dimensional or three dimensional) at the tracked position.”).
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider textures as claimed.
One of ordinary skill in the art would have motivation to combine in order to map the relationship between the effect object and tracked features in the scene. One of ordinary skill in the art would have had a reasonable expectation of success because Takada considers tracking a sequences of vertices in order to apply effects.
Allowable Subject Matter
Claim(s) 4-6, 8-9, 11 objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim(s) 4 , Takada considers:
collecting a trajectory point of the target object from an image frame of the target video according to a predetermined sampling frequency (Takada, ¶ 50: “As can be seen, the tracking times using the object pattern tracking module 122 are significantly higher for small tracked object (i.e., when the feature size in pixels is below 100, for example). As the feature size increases past this threshold, the tracking time for the template tracking module 123 also increases and becomes greater than the tracking time of the object pattern tracking module 122 for the same object.”);
Takada does not disclose the distance based threshold as claimed.
Regarding claim(s) 5, Takada does not suggest use of splines in this context.
Regarding claim(s) 6, Takada does not suggest dot product based grouping as claimed in the context of the independent claim. ,
Regarding claim(s) 8, Takada does not suggest “converting the boundary value of the three-dimensional topological structure to a calculated value of a percentage coordinate system of an image” in this context.
Regarding claim(s) 9, Takada does not suggest the claimed texture map sequence. Regarding claim(s) 11, Takada does not suggest setting the channel to zero in the context of parent claim 10.
Additional Prior Art
Additional prior art relevant to Applicant’s disclosure but not relied upon:
Tian (US 2025/0113005) also considers trajectory tracking:
PNG
media_image3.png
316
530
media_image3.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN M GRAY whose telephone number is (571)272-4582. The examiner can normally be reached on Monday through Friday, 9:00am-5:30pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached on (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN M GRAY/Primary Examiner, Art Unit 2611
Prosecution Insights