Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments and amendment have persuasively overcome the 112(a), 112(b) and 102 rejections. The remaining issues are addressed below.
The 101 subject matter eligibility rejection has been removed because of the amendments regarding the node architecture. However, if these limitations are removed (perhaps due to a lack of written description support), the examiner expects to re-apply the 101 subject matter eligibility rejection.
Examiner Note
While the present claims have introduced new limitations directed to nodes and node architectures, at present they are not considered a search burden. However, further amendments directed to nodes or node architectures may constitute a search burden such that those claims are non-responsive under election by original presentation. MPEP 821.
Claim Objections
Claims 1, 8, and 15 objected to because of the following informalities:
Claims 1, 8, and 15 recite “point interest,” but omit the “of.”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claims 1, 8, and 15 recite a node architecture and various nodes. Rather than a new matter rejection, this is a lack of written description rejection because the specification does not provide sufficient detail to convey that the inventors had possession of this idea. At the outset, while the claim now specifies what is meant by a “node,” the examiner has not identified similar usage of this term in the prior art. As explained by MPEP 2163(I)(A), “The claimed invention as a whole may not be adequately described if the claims require an essential or critical feature which is not adequately described in the specification and which is not conventional or known in the art.” Here, the examiner has not found anything conventional or known that the specification is basing its disclosure on, and thus Applicant has to fully describe all of the claimed node architecture and various nodes. However, the specification does not even describe what a node is, only a few examples of what is achieved. As examples, the specification does not convey what a recognizer node is (e.g., is this a machine learning technique, if so, which one?) Whatever the various nodes are, how does a node authoring engine create them? Note that providing written description support requires more than simply answering these questions, rather these questions are chosen to highlight the lack of support. For example, showing possession of a particular neural network requires providing details that demonstrate that the inventors had reduced the invention to practice, and given the wide variations in performance due to training and architecture, one would expect performance results and details on the architecture and training.
Additionally, claiming that a node “is configured to perform a defined operation,” without specifying what the operation might be is interpreted to cover any and all operations.
Dependent claims are likewise rejected.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 1, 8, and 15 recite “node architecture” and various “nodes,” but these are new terminology. MPEP 2173.05(a). While the claim conveys the general concept that this is some amount of software that performs some function, this general concept is not definite. Rather, it is unclear, for example, if these “nodes” and “node architecture” are an entirely new programming paradigm, or if they encompass existing technologies. The examiner’s best guess is that a recognizer node involves a specially trained convolutional neural network, but the specification does not disclose convolution, or even that a recognizer node uses a neural network. Further, is there a minimum number of nodes for something to qualify as a node architecture? Do the nodes need to be connected in a particular way to be a node architecture?
Claims 1, 8, and 15 recite “corresponds to a region,” but this is subjective. MPEP 2173.05(a). Specifically, if a point of interest is intended to be in a region, but is not, does that count as corresponding? It is also unclear if the “region” is limited to where the object is (i.e., the exact outline of the object), or if the “region” is understood as a region of interest (e.g., a bounding box around the object).
Dependent claims are likewise rejected.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 (all claims) are rejected under 35 U.S.C. 103 as being unpatentable over U.S. Pat. 9358456 (“Challinor”) in view of the Scratch programming language, website, and article “Scratch (programming language),” Wikipedia, March 19, 2023, retrieved from https://en.wikipedia.org/w/index.php?title=Scratch_(programming_language)&oldid=1145425560 (collectively, “Scratch”). Citations are to the Wikipedia page, but future addition of other references regarding Scratch as prior art would be an example of elaborating on the Scratch reference. MPEP 1207.03(a)(II).
Examiner Note
MPEP 2111.05(III) directs “However, where the claim as a whole is directed to conveying a message or meaning to a human reader independent of the intended computer system, and/or the computer-readable medium merely serves as a support for information or data, no functional relationship exists.” Here, all of the claim elements related to “visual effects” are printed matter and thus not entitled to patentable weight. In the interest of compact prosecution, the Office Action has mapped them anyway.
The broadest reasonable interpretation of “location” includes interpreting it to be the entire screen, e.g., “the effect is located on the screen.”
1. A method comprising:
receiving an image including an object having a pose, the pose defined by an orientation and a position of one or more points of interest, wherein each point interest corresponds to a region of the object within the image; (Challinor, claim 1 “receive information from a camera system reflecting a position of the first player in response to the first prompt”)
generating a set of key joint data for the object, wherein the set of key joint data includes the orientation and the position of the one or more points of interest of the object within a coordinate space; (Challinor, Figs. 1B and 1C)
creating a vector representation of the set of key joint data; (Challinor, Figs. 1B and 1C)
the node architecture including a recognizer node configured to activate a pose-based trigger when poses matching the pose of the object in the received image are detected by the recognizer node; (Challinor, Figs. 1B and 1C. Figs. 4 and 5 show how the skeletons are tracked into dance moves. Scratch, the below secondary reference, teaches the claimed “node architecture” and use of nodes generically.)
receiving one or more additional images that each include a candidate pose; (Challinor, claim 1 “receive information from the camera system reflecting a position of the second player in response to the second prompt”)
detecting, by the recognizer node, a match between one of the candidate poses in the one or more additional images and the pose of the received image by computing a similarity between the vector representation of the set of key joint data and respective vector representations for each of the candidate poses; and (Challinor, claim 1 “compare, by the machine, the input frame to the target frame to determine a comparison value” Scratch, the below secondary reference, teaches the claimed “node architecture” and use of nodes generically.)
rendering, by a render node of the node architecture, an output image with a visual effect applied to a location of at least one key joint within coordinate space based on the match. (Challinor, Fig. 14. See also, 14:65-15:3, “To assist the user in completing moves correctly, per-limb feedback can be given to the user when performing a move. In some embodiments, if the user is not satisfying a filter for a limb, the game can render a red outline around the on-screen dancer's corresponding limb to demonstrate to the user where they need to make an adjustment.” Scratch, the below secondary reference, teaches the claimed “node architecture” and use of nodes generically.)
Challinor is not relied on for the below claim language.
However, Scratch teaches generating a node architecture based on a received selection of nodes, wherein each node of the node architecture is configured to perform a defined operation, (Scratch, first section, “Users on the site, called Scratchers, can create projects on the website using a block-like interface. Projects can be exported to standalone HTML5, Android apps, Bundle (macOS) and EXE files using external tools.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Scratch to the teachings of Challinor such that Scratch’s visual programming can control Challinor’s game for the purpose of remixing video games (Scratch, first section). Additionally, Challinor discusses developers of the game, such as “Beneficially, the game developer can manually weight certain moves to be more forgiving along any axis simply because a move is hard to perform.” Challinor, 14:39-41. Scratch provides implementation details for this and Scratch creates a wider pool of developers because it has wide outreach and it at least used to be a top 20 programming language. Scratch, section “Community of users.” (A wider pool of developers makes Challinor’s system more valuable.)
Based on the above, this is an example of “combining prior art elements according to known methods to yield predictable results.” MPEP 2143.
2. The method of claim 1, wherein generating a set of key joint data for the object comprises:
applying a trained machine learning model to the image including the object, (Challinor, 4:5 “FIG. 1B depicts an example of a skeleton provided by a MICROSOFT KINECT” Microsoft Kinect uses machine learning, see, e.g., https://people.math.harvard.edu/archive/21a_fall_13/exhibits/mackormick/kinect.pdf, slide 4 [also attached as an appendix to the June 9, 2025 Office Action])
wherein applying the trained machine learning model comprises:
detecting a type of the object, the type indicating a set of points that are defined for the object; (Challinor, 4:5 “FIG. 1B depicts an example of a skeleton provided by a MICROSOFT KINECT” Challinor’s skeleton teaches the claimed type of the object.)
detecting a position and orientation for each point in the set of points; and (Challinor, Figs. 1B and 1C)
inserting the position and orientation of each point into the set of key joint data. (Challinor, Figs. 1B and 1C)
3. The method of claim 2, wherein detecting a match between one of the candidate poses in one or more additional images and the pose of the object in the received image comprises:
for each candidate pose of the candidate poses (This claim reads on a single candidate pose)
comparing a key joint of the pose of the object within the image to a corresponding key joint of a candidate pose of a candidate object that is of a same type as the type of the object; and (Challinor, Fig. 12)
The examiner notes that while the above claim element recites “a candidate pose,” “the” may have been intended.
determining, based on the comparing, that the key joint of the pose matches the corresponding key joint of the candidate pose. (Challinor, Fig. 12)
4. The method of claim 3, wherein rendering the output image with the visual effect applied to the location of the at least one key joint within the coordinate space based on the match comprises:
selecting the visual effect for insertion into the image; and (Challinor, Fig. 14. Fig. 14 shows that the score is a visual effect, claim 2 states that the score is a result of the comparison value (as above, the comparison value teaches the claimed detecting a match))
in response to determining, based on the comparing, that key joint of the pose matches the corresponding key joint of the candidate pose, inserting the selected visual effect into the image. (Challinor, Fig. 14. Challinor’s increased score teaches the claimed selected visual effect.)
5. The method of claim 4, inserting the selected visual effect into the image comprises:
identifying at least one key joint of the pose where the visual effect is to be added; and (Challinor, Fig. 14. Challinor’s animated dancer teaches the claimed visual effect, the effects correspond to where the player is)
applying the visual effect at a position of the at least one key joint in the image. (Challinor, Fig. 14. Challinor’s animated dancer teaches the claimed visual effect)
6. The method of claim 1 further comprising:
receiving a second image including the object having an additional pose, the additional pose defined by an additional orientation and an additional position of the object within the image; (Challinor, Fig. 9, “Has the player performed the dance move four times?” 915)
generating an additional set of key joint data that represents the additional orientation and additional position of the one or more points of interest for the object; (Challinor, Fig. 9, “Capture the dance move with the camera and provide corresponding skeleton to game platform” 910. Challinor’s skeleton teaches the claimed key joint data, orientation, position and points of interest.)
creating a vector representation of the additional set of key joint data; (Challinor, Fig. 9, “Capture the dance move with the camera and provide corresponding skeleton to game platform” 910. Challinor’s skeleton teaches the claimed vector representation.)
in response to receiving the one or more additional images, detecting an occurrence of the pose at a first time; (Challinor, Fig. 9, second performance 925)
in response to receiving the one or more additional images, detecting an occurrence of the additional pose at a second time; and (Challinor, Fig. 9, third performance 925)
rendering an output image with a second visual effect based on the occurrence of the pose and the additional pose. (Challinor, 34:55-34:63 “In some embodiments, the game platform is configured (e.g., vis-à-vis computer source code) to create icons such as flashcards that can be used to instruct the second player how to perform the first player's dance move. As described more fully above (e.g., with respect to FIGS. 5-7), one or more icons can be displayed at one or more fixed or predetermined locations on the display. For example, the icons can be displayed as previous, current, and next dance move as described above.”)
7. The method of claim 6, wherein detecting an occurrence of the pose at a first time comprises comparing the vector representation of the additional set of key joint data to the set of key joint data that represents the orientation and position of one or more points of interest associated with the object. (Challinor, Fig. 9, “Are the three performances similar?”)
Claims 8-20 are rejected for the same rationale as their counterpart claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
U.S. Pat. Pub. 20230010480 (“Li”) – Fig. 11 anticipates the original claim 1, see, e.g., [0086]-[0089]
U.S. Pat. 8241118 – titled “System for promoting physical activity employing virtual interactive arena”
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID ORANGE whose telephone number is (571)270-1799. The examiner can normally be reached Mon-Fri, 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at 571-272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ORANGE/Primary Examiner, Art Unit 2663
Prosecution Insights