DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-15 are pending.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 3/27/24, 11/14/24, and 1/27/25 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Specification
The summary of the invention should be separate and distinct from the abstract. See section (h) below.
Content of Specification
(a) TITLE OF THE INVENTION: See 37 CFR 1.72(a) and MPEP § 606. The title of the invention should be placed at the top of the first page of the specification unless the title is provided in an application data sheet. The title of the invention should be brief but technically accurate and descriptive, preferably from two to seven words. It may not contain more than 500 characters.
(b) CROSS-REFERENCES TO RELATED APPLICATIONS: See 37 CFR 1.78 and MPEP § 211 et seq.
(c) STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT: See MPEP § 310.
(d) THE NAMES OF THE PARTIES TO A JOINT RESEARCH AGREEMENT. See 37 CFR 1.71(g).
(e) INCORPORATION-BY-REFERENCE OF MATERIAL SUBMITTED ON A READ-ONLY OPTICAL DISC, AS A TEXT FILE OR AN XML FILE VIA THE PATENT ELECTRONIC SYSTEM: The specification is required to include an incorporation-by-reference of electronic documents that are to become part of the permanent United States Patent and Trademark Office records in the file of a patent application. See 37 CFR 1.77(b)(5) and MPEP § 608.05. See also the Legal Framework for Patent Electronic System posted on the USPTO website (https://www.uspto.gov/sites/default/files/documents/2019LegalFrameworkPES.pdf) and MPEP § 502.05
(f) STATEMENT REGARDING PRIOR DISCLOSURES BY THE INVENTOR OR A JOINT INVENTOR. See 35 U.S.C. 102(b) and 37 CFR 1.77.
(g) BACKGROUND OF THE INVENTION: See MPEP § 608.01(c). The specification should set forth the Background of the Invention in two parts:
(1) Field of the Invention: A statement of the field of art to which the invention pertains. This statement may include a paraphrasing of the applicable U.S. patent classification definitions of the subject matter of the claimed invention. This item may also be titled “Technical Field.”
(2) Description of the Related Art including information disclosed under 37 CFR 1.97 and 37 CFR 1.98: A description of the related art known to the applicant and including, if applicable, references to specific related art and problems involved in the prior art which are solved by the applicant’s invention. This item may also be titled “Background Art.”
(h) BRIEF SUMMARY OF THE INVENTION: See MPEP § 608.01(d). A brief summary or general statement of the invention as set forth in 37 CFR 1.73. The summary is separate and distinct from the abstract and is directed toward the invention rather than the disclosure as a whole. The summary may point out the advantages of the invention or how it solves problems previously existent in the prior art (and preferably indicated in the Background of the Invention). In chemical cases it should point out in general terms the utility of the invention. If possible, the nature and gist of the invention or the inventive concept should be set forth. Objects of the invention should be treated briefly and only to the extent that they contribute to an understanding of the invention.
(i) BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING(S): See MPEP § 608.01(f). A reference to and brief description of the drawing(s) as set forth in 37 CFR 1.74.
(j) DETAILED DESCRIPTION OF THE INVENTION: See MPEP § 608.01(g). A description of the preferred embodiment(s) of the invention as required in 37 CFR 1.71. The description should be as short and specific as is necessary to describe the invention adequately and accurately. Where elements or groups of elements, compounds, and processes, which are conventional and generally widely known in the field of the invention described, and their exact nature or type is not necessary for an understanding and use of the invention by a person skilled in the art, they should not be described in detail. However, where particularly complicated subject matter is involved or where the elements, compounds, or processes may not be commonly or widely known in the field, the specification should refer to another patent or readily available publication which adequately describes the subject matter.
(k) CLAIM OR CLAIMS: See 37 CFR 1.75 and MPEP § 608.01(m). The claim or claims must commence on a separate sheet or electronic page (37 CFR 1.52(b)(3)). Where a claim sets forth a plurality of elements or steps, each element or step of the claim should be separated by a line indentation. There may be plural indentations to further segregate subcombinations or related steps. See 37 CFR 1.75 and MPEP 608.01(i) - (p).
(l) ABSTRACT OF THE DISCLOSURE: See 37 CFR 1.72 (b) and MPEP § 608.01(b). The abstract is a brief narrative of the disclosure as a whole, as concise as the disclosure permits, in a single paragraph preferably not exceeding 150 words, commencing on a separate sheet following the claims. In an international application which has entered the national stage (37 CFR 1.491(b)), the applicant need not submit an abstract commencing on a separate sheet if an abstract was published with the international application under PCT Article 21. The abstract that appears on the cover page of the pamphlet published by the International Bureau (IB) of the World Intellectual Property Organization (WIPO) is the abstract that will be used by the USPTO. See MPEP § 1893.03(e).
(m) SEQUENCE LISTING: See 37 CFR 1.821 - 1.825 and MPEP §§ 2421 - 2431. The requirement for a sequence listing applies to all sequences disclosed in a given application, whether the sequences are claimed or not. See MPEP § 2422.01.
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “control unit configured to perform” in claims 14 and 15.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4-7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 recites the limitation "degree of smoothing" in line 4. It is unclear and indefinite if this is the same as the “degree of smoothing” previously recited in claim 3, from which claim 4 depends.
Claim 5 recites the limitation "smoothing" in line 3. It is unclear and indefinite if this is the same as the “smoothing” previously recited in the claim.
Claim 6 recites the limitation "smoothing" in line 3. It is unclear and indefinite if this is the same as the “smoothing” previously recited in the claim.
Claim 7 recites the limitation "part" in line 2. It is unclear and indefinite if this is referring to the “part of the skeleton” or the “object part” previously recited in claim 1, from which claim 7 depends.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite a non-transitory computer-readable recording medium, method, and apparatus for processing skeleton data of an object. With respect to the analysis of claims 12 and 13 (claims 1 and 14 recite similar limitations to claim 12, and claims 11 and 15 recite similar limitations to claim 13):
Step 1:
With regard to Step 1, the claims are directed to a method; and therefore, the claims are directed to one of the statutory categories of inventions.
Step 2A, Prong One:
With regard to Step 2A, Prong One, the limitations in claim 12 “calculating a feature amount that represents a temporal change of a movement for each part of the skeleton of the object based on the acquired skeleton data; and specifying an object part to be smoothed in the skeleton represented by the skeleton data based on the calculated feature amount” as drafted, recite an abstract idea, such as a mathematical calculation and a process that, under its broadest reasonable interpretation, covers performance of the limitation manually or in the mind by a human. That is, a person can calculate a change in a person’s movement for a specific part/joint of the person’s body and determine which part/joint should be smoothed by a filter.
These are concepts that fall under the grouping of abstract idea mathematical calculations and mental processes, i.e., a concept performed in the human mind, evaluation, judgment, and/or opinion of a human.
With regard to Step 2A, Prong One, the limitations in claim 13 “calculating a feature amount that represents a temporal change of a movement for each part of the skeleton of the object based on the acquired skeleton data; and specifying a coefficient of a smoothing filter to be applied to a part of the skeleton represented by the skeleton data based on the calculated feature amount” as drafted, recite an abstract idea, such as a mathematical calculation and a process that, under its broadest reasonable interpretation, covers performance of the limitation manually or in the mind by a human. That is, a person can calculate a change in a person’s movement for a specific part/joint of the person’s body and determine/select the smoothing filter coefficient to apply to the part/joint. These are concepts that fall under the grouping of abstract idea mathematical calculations and mental processes, i.e., a concept performed in the human mind, evaluation, judgment, and/or opinion of a human.
Step 2A, Prong Two:
The 2019 PEG defines the phrase “integration into a practical application” to require an additional element or a combination of additional elements in the claim to apply, rely on, or use the judicial exception. In the instant case, there are no additional steps/elements/limitations in the claims, with the exception of the following in the claims: “computer” and “acquiring skeleton data that represents, in time series, a skeleton of an object recognized based on sensor data for the object” in claims 1 and 11-13, and “control unit” and “acquiring skeleton data that represents, in time series, a skeleton of an object recognized based on sensor data for the object” in claims 14 and 15. The acquiring step is data gathering/data input. The computer and control unit are generic computer components. These limitations are regarded as adding routine and conventional elements to perform the judicial exception, and do not apply into a practical application. Accordingly, the above-mentioned additional elements/limitations do not integrate the abstract idea into a practical application; and therefore, the claims recite an abstract idea.
Step 2B:
Because the claims fail under Step 2A, the claims are further evaluated under Step 2B. The claims herein do not include additional elements that are sufficient to amount to significantly more than the judicial exception, because as discussed above with respect to integration of the abstract idea into practical application, the additional elements/limitations to perform the steps, amount to no more than insignificant routine and conventional elements. Mere instructions to apply an exception using generic components cannot provide an inventive concept. Therefore, claims 1 and 11-15 are not patent eligible.
Furthermore, with regard to Claims 2-10 viewed individually, these additional steps, under their broadest reasonable interpretation, provide extra-solution activities to cover performance of the limitations as an abstract idea, and do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Accordingly, they are not patent eligible.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 7-10, 12, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Iwao (US 2020/0005670 A1).
Regarding claim 1, Iwao teaches, A non-transitory computer-readable recording medium storing an information processing program for causing a computer to execute processing comprising (Para. 0021: CPU executes programs stored in the storage unit to perform various processing; Para. 0075):
acquiring skeleton data that represents, in time series, a skeleton of an object recognized based on sensor data for the object (Para. 0025: a learner (i.e., object or individual trying to achieve the motion of another person) is located in a space and cameras are installed to image the object; Para. 0028: camera group captures moving images of the learner’s swing; Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0058: start and end time of the motion of the learner);
calculating a feature amount that represents a temporal change of a movement for each part of the skeleton of the object based on the acquired skeleton data (Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0046: “joints of the head, the neck, the shoulders, elbows, wrists, a vicinity of a pelvis, thighs, knees, and ankles are calculated”; Para. 0058: start and end time of the motion of the learner);
and specifying an object part to be smoothed in the skeleton represented by the skeleton data based on the calculated feature amount (Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0046: “joints of the head, the neck, the shoulders, elbows, wrists, a vicinity of a pelvis, thighs, knees, and ankles are calculated”; Para. 0057: Moving image representing motion of a golf swing by the learner is displayed and there are inflection points in the skeleton motion. When the skeleton motion of the learner based on the captured data includes noise, smoothing processing is performed. The smoothing processing can be performed on all joints).
Iwao discloses and teaches the above limitations in different embodiments. It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine the embodiments for acquiring skeleton data, calculating a feature amount representing a temporal change of a movement for each part of the skeleton of the object, and specifying an object part to be smoothed in the skeleton based on the calculated feature amount since the embodiments/features can be combined (Iwao, Para. 0019), and in order to eliminate noise when processing the joints (Iwao, Para. 0057). Therefore, one of ordinary skill in the art would be capable to have combined the embodiments as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned reasons that the Examiner has reached a conclusion of obviousness with respect to claim 1.
Regarding claim 5, Iwao teaches the limitations as explained above in claim 1.
Iwao further teaches, The non-transitory computer-readable recording medium according to claim 1 (see claim 1 above), for causing the computer to further execute processing of (Para. 0021: CPU executes programs stored in the storage unit to perform various processing; Para. 0075), when smoothing processing for the skeleton data is performed, performing smoothing for the specified object part in the skeleton represented by the skeleton data (Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0046: “joints of the head, the neck, the shoulders, elbows, wrists, a vicinity of a pelvis, thighs, knees, and ankles are calculated”; Para. 0057: Moving image representing motion of a golf swing by the learner is displayed and there are inflection points in the skeleton motion. When the skeleton motion of the learner based on the captured data includes noise, smoothing processing is performed. The smoothing processing can be performed on all joints).
Regarding claim 7, Iwao teaches the limitations as explained above in claim 1.
Iwao further teaches, The non-transitory computer-readable recording medium according to claim 1 (see claim 1 above), wherein the part includes a plurality of joint points in a joint point group that forms the skeleton of the object (Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0046: “joints of the head, the neck, the shoulders, elbows, wrists, a vicinity of a pelvis, thighs, knees, and ankles are calculated, and the skeleton data is generated so as to connect the joints”; Para. 0047: joint of an arm; Fig. 5: elements 500/501 shows that the object’s/learner’s head, arms, legs, etc. have multiple joint points).
Regarding claim 8, Iwao teaches the limitations as explained above in claim 1.
Iwao further teaches, The non-transitory computer-readable recording medium according to claim 1 (see claim 1 above), wherein the skeleton data is time-series data that indicates a temporal change of a position of each joint point that forms the skeleton of the object in a three-dimensional space (Para. 0025: a learner (i.e., object or individual trying to achieve the motion of another person) is located in a space; Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0047: three-dimensional position/distance; Para. 0058: start and end time of the motion of the learner; Fig. 11 and Paras. 0051-0052: there is an initial position of the learner and then a position after a rotation of the joint; Para. 0053: there can also be three-dimensional rotation of the joint).
Regarding claim 9, Iwao teaches the limitations as explained above in claim 1.
Iwao further teaches, The non-transitory computer-readable recording medium according to claim 1 (see claim 1 above), wherein the object is a performer (Para. 0002: sports participants (“learners”) can be golfer; Para. 0025: sports and acting performances can be analyzed, including a golf swing; Para. 0057: motion of a golf swing by a learner; Note: the Examiner interprets a golfer as a performer (i.e., performer in golf)).
Regarding claim 10, Iwao teaches the limitations as explained above in claim 1.
Iwao further teaches, The non-transitory computer-readable recording medium according to claim 1 (see claim 1 above),
wherein the sensor data includes an image obtained by capturing the object (Para. 0025: a learner (i.e., object or individual trying to achieve the motion of another person) is located in a space and cameras are installed to image the object; Para. 0028: camera group captures moving images of the learner’s swing),
and in the processing of acquiring, by generating the skeleton data that represents the skeleton of the object recognized from the image included in the sensor data, the generated skeleton data is acquired (Para. 0025: a learner (i.e., object or individual trying to achieve the motion of another person) is located in a space and cameras are installed to image the object; Para. 0028: camera group captures moving images of the learner’s swing; Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0046).
Regarding claim 12, Iwao teaches, An information processing method implemented by a computer, the method comprising (Para. 0021: CPU executes programs stored in the storage unit to perform various processing; Para. 0046: method of estimating the skeleton data from moving images; Para. 0075):
acquiring skeleton data that represents, in time series, a skeleton of an object recognized based on sensor data for the object (Para. 0025: a learner (i.e., object or individual trying to achieve the motion of another person) is located in a space and cameras are installed to image the object; Para. 0028: camera group captures moving images of the learner’s swing; Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0058: start and end time of the motion of the learner);
calculating a feature amount that represents a temporal change of a movement for each part of the skeleton of the object based on the acquired skeleton data (Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0046: “joints of the head, the neck, the shoulders, elbows, wrists, a vicinity of a pelvis, thighs, knees, and ankles are calculated”; Para. 0058: start and end time of the motion of the learner);
and specifying an object part to be smoothed in the skeleton represented by the skeleton data based on the calculated feature amount (Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0057: Moving image representing motion of a golf swing by the learner is displayed and there are inflection points in the skeleton motion. When the skeleton motion of the learner based on the captured data includes noise, smoothing processing is performed. The smoothing processing can be performed on all joints).
Iwao discloses and teaches the above limitations in different embodiments. It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine the embodiments for acquiring skeleton data, calculating a feature amount representing a temporal change of a movement for each part of the skeleton of the object, and specifying an object part to be smoothed in the skeleton based on the calculated feature amount since the embodiments/features can be combined (Iwao, Para. 0019), and in order to eliminate noise when processing the joints (Iwao, Para. 0057). Therefore, one of ordinary skill in the art would be capable to have combined the embodiments as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned reasons that the Examiner has reached a conclusion of obviousness with respect to claim 12.
Regarding claim 14, Iwao teaches, An information processing apparatus comprising a control unit configured to perform processing comprising (Para. 0020 and Fig. 1: information processing apparatus 100 includes a central processing unit (CPU) 101; Para. 0021: CPU executes programs stored in the storage unit to perform various processing):
acquiring skeleton data that represents, in time series, a skeleton of an object recognized based on sensor data for the object (Para. 0025: a learner (i.e., object or individual trying to achieve the motion of another person) is located in a space and cameras are installed to image the object; Para. 0028: camera group captures moving images of the learner’s swing; Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0058: start and end time of the motion of the learner);
calculating a feature amount that represents a temporal change of a movement for each part of the skeleton of the object based on the acquired skeleton data (Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0046: “joints of the head, the neck, the shoulders, elbows, wrists, a vicinity of a pelvis, thighs, knees, and ankles are calculated”; Para. 0058: start and end time of the motion of the learner);
and specifying an object part to be smoothed in the skeleton represented by the skeleton data based on the calculated feature amount (Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0057: Moving image representing motion of a golf swing by the learner is displayed and there are inflection points in the skeleton motion. When the skeleton motion of the learner based on the captured data includes noise, smoothing processing is performed. The smoothing processing can be performed on all joints).
Iwao discloses and teaches the above limitations in different embodiments. It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine the embodiments for acquiring skeleton data, calculating a feature amount representing a temporal change of a movement for each part of the skeleton of the object, and specifying an object part to be smoothed in the skeleton based on the calculated feature amount since the embodiments/features can be combined (Iwao, Para. 0019), and in order to eliminate noise when processing the joints (Iwao, Para. 0057). Therefore, one of ordinary skill in the art would be capable to have combined the embodiments as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned reasons that the Examiner has reached a conclusion of obviousness with respect to claim 14.
Claims 11, 13, and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Iwao (US 2020/0005670 A1) in view of “The effects of using a noise filter and feature selection in action recognition: an empirical study” by Mendez et al. (hereinafter “Mendez”).
Regarding claim 11, Iwao teaches, A non-transitory computer-readable recording medium storing an information processing program for causing a computer to execute processing comprising (Para. 0021: CPU executes programs stored in the storage unit to perform various processing; Para. 0075):
acquiring skeleton data that represents, in time series, a skeleton of an object recognized based on sensor data for the object (Para. 0025: a learner (i.e., object or individual trying to achieve the motion of another person) is located in a space and cameras are installed to image the object; Para. 0028: camera group captures moving images of the learner’s swing; Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0058: start and end time of the motion of the learner);
calculating a feature amount that represents a temporal change of a movement for each part of the skeleton of the object based on the acquired skeleton data (Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0046: “joints of the head, the neck, the shoulders, elbows, wrists, a vicinity of a pelvis, thighs, knees, and ankles are calculated”; Para. 0058: start and end time of the motion of the learner);
and specifying (Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0057: Moving image representing motion of a golf swing by the learner is displayed and there are inflection points in the skeleton motion. When the skeleton motion of the learner based on the captured data includes noise, smoothing processing is performed. The smoothing processing can be performed on all joints and may be a mean value filter, a median filter, and a Gaussian filter).
Iwao does not expressly disclose the following limitation: specifying a coefficient of a smoothing filter.
However, Mendez teaches, specifying a coefficient of a smoothing filter (Pg. 44, section III. Filter Phase: The Savitzky-Golay filter is used with coefficients to smooth the skeleton frame; Fig. 2: in each of the six frames, in the middle is the skeleton of the Kalman filter; Pg. 46 and Tables I and II: skeletal joints positions were pre-processed using the Kalman filter (KF) and Savitzky-Golay filter (SGF) for selected features).
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine specifying a coefficient of a smoothing filter as taught by Mendez with the method of Iwao in order to improve action recognition (Mendez, Pg. 44, section II. Proposed System). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 11.
Regarding claim 13, Iwao teaches, An information processing method implemented by a computer, the method comprising (Para. 0021: CPU executes programs stored in the storage unit to perform various processing; Para. 0046: method of estimating the skeleton data from moving images; Para. 0075):
acquiring skeleton data that represents, in time series, a skeleton of an object recognized based on sensor data for the object (Para. 0025: a learner (i.e., object or individual trying to achieve the motion of another person) is located in a space and cameras are installed to image the object; Para. 0028: camera group captures moving images of the learner’s swing; Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0058: start and end time of the motion of the learner);
calculating a feature amount that represents a temporal change of a movement for each part of the skeleton of the object based on the acquired skeleton data (Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0046: “joints of the head, the neck, the shoulders, elbows, wrists, a vicinity of a pelvis, thighs, knees, and ankles are calculated”; Para. 0058: start and end time of the motion of the learner);
and specifying (Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0057: Moving image representing motion of a golf swing by the learner is displayed and there are inflection points in the skeleton motion. When the skeleton motion of the learner based on the captured data includes noise, smoothing processing is performed. The smoothing processing can be performed on all joints and may be a mean value filter, a median filter, and a Gaussian filter).
Iwao does not expressly disclose the following limitation: specifying a coefficient of a smoothing filter.
However, Mendez teaches, specifying a coefficient of a smoothing filter (Pg. 44, section III. Filter Phase: The Savitzky-Golay filter is used with coefficients to smooth the skeleton frame; Fig. 2: in each of the six frames, in the middle is the skeleton of the Kalman filter; Pg. 46 and Tables I and II: skeletal joints positions were pre-processed using the Kalman filter (KF) and Savitzky-Golay filter (SGF) for selected features).
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine specifying a coefficient of a smoothing filter as taught by Mendez with the method of Iwao in order to improve action recognition (Mendez, Pg. 44, section II. Proposed System). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 13.
Regarding claim 15, Iwao teaches, An information processing apparatus comprising a control unit configured to perform processing comprising (Para. 0020 and Fig. 1: information processing apparatus 100 including a central processing unit (CPU) 101; Para. 0021: CPU executes programs stored in the storage unit to perform various processing):
acquiring skeleton data that represents, in time series, a skeleton of an object recognized based on sensor data for the object (Para. 0025: a learner (i.e., object or individual trying to achieve the motion of another person) is located in a space and cameras are installed to image the object; Para. 0028: camera group captures moving images of the learner’s swing; Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0058: start and end time of the motion of the learner);
calculating a feature amount that represents a temporal change of a movement for each part of the skeleton of the object based on the acquired skeleton data (Para. 0029: skeleton data of the learner is estimated based on the captured images. The skeleton data about the object represents a bone structure of the object and is expressed by markers indicating positions of a plurality of joints of the object and line segments connecting the markers; Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0039: motion information of the skeleton represents the motion of the plurality of joints of the object; Para. 0046: “joints of the head, the neck, the shoulders, elbows, wrists, a vicinity of a pelvis, thighs, knees, and ankles are calculated”; Para. 0058: start and end time of the motion of the learner);
and specifying (Para. 0031: change of an angle of a specific joint in each object with time and an inflection point in the change of the angle; Figs. 3 and 5: graph of the change of angle of a joint of the learner over time; Para. 0057: Moving image representing motion of a golf swing by the learner is displayed and there are inflection points in the skeleton motion. When the skeleton motion of the learner based on the captured data includes noise, smoothing processing is performed. The smoothing processing can be performed on all joints and may be a mean value filter, a median filter, and a Gaussian filter).
Iwao does not expressly disclose the following limitation: specifying a coefficient of a smoothing filter.
However, Mendez teaches, specifying a coefficient of a smoothing filter (Pg. 44, section III. Filter Phase: The Savitzky-Golay filter is used with coefficients to smooth the skeleton frame; Fig. 2: in each of the six frames, in the middle is the skeleton of the Kalman filter; Pg. 46 and Tables I and II: skeletal joints positions were pre-processed using the Kalman filter (KF) and Savitzky-Golay filter (SGF) for selected features).
It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art to combine specifying a coefficient of a smoothing filter as taught by Mendez with the method of Iwao in order to improve action recognition (Mendez, Pg. 44, section II. Proposed System). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 15.
Allowable Subject Matter
Claims 2 and 3 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Claims 4 and 6 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 101 and U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Tanabiki et al. (US 2013/0230211 A1)
Yabuki et al. (US 2019/0220657 A1)
Fujimoto et al. (US 2019/0066327 A1)
Fukumoto et al. (US 2021/0166479 A1)
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daniella M. DiGuglielmo whose telephone number is (571)272-0183. The examiner can normally be reached Monday - Friday 8:00 AM - 4:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daniella M. DiGuglielmo/Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666