DETAILED ACTION
Remarks
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 4 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention. Claim 4 recites the limitation “the second device auxiliary” which lacks sufficient antecedent basis in the claims.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-9 and 12-15 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Under the broadest reasonable interpretation, the terms of the claim are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP 2111.
STEP 1 = YES: The claimed invention is to a process and product, and thus fall under one of the four statutory categories (Step 1: YES).
STEP 2A, Prong 1 = YES: The claim(s) recite(s) a series of steps which can be practically performed by one or more humans through mental process (i.e., observation, evaluation, judgement, and/or opinion)(see MPEP § 2106.04(a)(2), subsection III) and/or certain methods of organizing human activity (i.e., managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions) (see MPEP § 2106.04(a)(2), subsection II). Moreover, the claims recite steps akin to “collecting information, analyzing it, and displaying certain results of the collection and analysis,” where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, which the court in Electric Power Group held to recite a mental process. Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016). This includes:
1. A system for assisting a user to learn foreign languages, including: …
…storing …exemplary non-verbal communication skills to be demonstrated by a speaker during conversation (mental process, evaluation);
… storing …evaluation model for evaluating non-verbal communication skills of a speaker during conversation (mental process, evaluation);
… comparing the exemplary non-verbal communication skills stored … to non-verbal communication skills of the user having been acquired … to thereby evaluate non-verbal communication skills of the user (mental process, evaluation); and
… displaying evaluation made … (mental process, evaluation; certain methods of organizing human activity, interaction between individuals, e.g., teaching).
2. The system as set forth in claim 1, …evaluates non-verbal communication skills of the user with respect to at least one items selected among countenance, gaze, gesture, body action, proxemics, physical appearance, visual focus, auditory information and cultural background (mental process, observation and evaluation).
3. The system as set forth in claim 2, … extracts a feature degree indicating quantitatively a feature of each of the items, based on image data of the user having been acquired …, and compares the thus extracted feature degree to the exemplary non-verbal communication skills stored … (mental process, evaluation).
4. The system as set forth in claim 1, … uses conversation of the user as verbal data in evaluation of the non-verbal communication skills of the user (mental process, evaluation).
5. The system as set forth in claim 1, further including: … storing …cultural background data of various countries and regions; and … reading cultural background data of the user…, and taking the cultural background data of the user into consideration in evaluation of non-verbal communication skills of the user … (mental process, evaluation).
6. The system as set forth in claim 1,…evaluate non-verbal communication skills of a speaker with the exemplary non-verbal communication skills…being used as criteria…the evaluation being made based on the exemplary non-verbal communication skills (mental process, evaluation).
7. The system as set forth in claim 1, further including …making curriculum specialized for the user so as to compensate for the non-verbal communication skills of the user having been low-evaluated … (mental process, evaluation).
8. The system as set forth in claim 7, further including … making learning materials in line with the curriculum … (mental process, evaluation).
9. The system as set forth in claim 8, wherein the learning materials include a moving picture in which characters and the user make conversation with each other (mental process, evaluation).
13. A method of assisting a user to learn foreign languages, including: …
comparing exemplary non-verbal communication skills to non-verbal communication skills of the user having been acquired in the picture-taking step… to thereby evaluate the non-verbal communication skills of the user, the exemplary non-verbal communication skills being read out of a memory storing therein exemplary non-verbal communication skills including exemplary countenance, gesture and so on to be demonstrated by a speaker in conversation (mental process, evaluation); and
displaying evaluation made in the comparison step (mental process, evaluation; certain methods of organizing human activity, interaction between individuals, e.g., teaching).
The steps identified above are akin to organizing human activity, mental processes, and/or mathematical concepts, and thus fall within an enumerated category of abstract ideas. Note that even if most humans would use a physical aid (e.g., pen and paper, a slide rule, or a calculator) to help them complete the recited steps above, the use of such physical aid does not negate the mental nature of these limitations. Therefore, the claims recite an abstract idea (Step 2A, Prong 1: YES).
STEP 2A, Prong 2 = NO: This judicial exception is not integrated into a practical application.
To the extent the claims recite additional elements related to defining a computer environment to implement the abstract idea above (i.e., a first and second memory storing; and further including a trained evaluation model, made by machine learning, wherein an input to the trained evaluation model includes image data of non-verbal communication skills of the user during conversation, the image data being taken by the first device, and an output from the trained evaluation model is evaluation to the non-verbal communication skills of the user during conversation, for performing the steps identified as an abstract idea under Prong 1, and performing the steps identified as an abstract idea under Prong 1 by means of a trained evaluation model used for evaluating non-verbal communication skills of a speaker in conversation; and further including a second device/device auxiliary for ; and further including a fourth, fifth, and sixth device for performing the steps identified as an abstract idea under Prong 1; and further including evaluating information out of the first database or in the first memory; 12. A portable wireless communication device including the system as set forth in claim 1; 14. A recording medium readable by a computer, storing a program therein for causing a computer to carry out the method as set forth in claim 13; 15. A portable wireless communication device including a program for causing the portable wireless communication device to carry out the method as set forth in claim 13), the specification does not identify technical improvements provided by these limitations. Further, these limitations are recited in the claims at a high level of generality such that they do not amount to a particular machine or technical improvement thereof, nor do they represent an improvement in any other technology. Further, the claims do not define the model or machine learning with any technical detail, e.g., adjusting values of parameters. Rather, the generic manner which these additional elements are claimed amount to mere instructions to implement the abstract idea in a computer environment, i.e., field of use, and thus do not integrate the judicial exception into a practical application.
To the extent the claims recite additional elements related to a physical component for providing data collection (i.e., a first device for taking a picture of the user pronouncing in accordance with audio of a moving picture; and taking a picture of the user pronouncing in accordance with audio of a moving picture), the claims do not recite a particular machine or other technical detail to perform these limitations. Rather, the claimed devices are merely recited to perform insignificant pre-solution data gathering activity, e.g., take a picture, which but for the generic recitation of physical devices, is practically capable of being performed by human analog, e.g., by mental observation. Therefore, the use of devices in the claims does not integrate the judicial exception into a practical application.
To the extent the claims recite additional elements related to a physical component for providing data output (i.e., a third device for displaying), the claims do not recite a particular manner of displaying information on the device. Rather, the claimed data output to a display are recited at a high level of generality without any technical detail defining either a particular machine nor an improvement in display technology. Thus, the claimed additional elements defining the presented data as via a display screen amount to insignificant post-solution data output activity. Therefore, the recitation of displaying information via a display in the claims does not integrate the judicial exception into a practical application.
It should be noted that because the courts have made it clear that mere physicality or tangibility of an additional element or elements is not a relevant consideration in the eligibility analysis, the physical nature of the physical components identified above does not affect this analysis. See MPEP 2106.05(I) for more information on this point, including explanations from judicial decisions including Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 573 U.S. 208, 224-26 (2014). Even when viewed in combination, these additional elements do not integrate the recited judicial exception into a practical application and the claim is directed to the judicial exception. Therefore, the claims are directed to an abstract idea (Step 2A, Prong 2: YES).
STEP 2B = NO: The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because as provided under Prong 2, the additional elements are recited at a high level of generality, and for the purpose of insignificant pre and post-solution activity. Moreover, the specification of the instant application further demonstrates that the additional elements are recited for their well-understood, routine and conventional functionality, which refers to elements of the computer system in a manner that indicates that the additional elements are sufficiently well-known that the specification does not need to describe the particulars of such additional elements to satisfy 35 U.S.C. § 112(a). Thus, the additional elements, as identified under Prong 2, amount to merely automating a manual process, which the courts have held to be insufficient in showing an improvement in computer-functionality. See Credit Acceptance Corp. v. Westlake Services, 859 F.3d 1044, 1055, 123 USPQ2d 1100, 1108-09 (Fed. Cir. 2017); see also LendingTree, LLC v. Zillow, Inc., 656 Fed. App'x 991, 996-97 (Fed. Cir. 2016) (non-precedential).
Therefore, the claims are not directed to significantly more than the abstract idea (Step 2B: NO).
Therefore, claims 1-9 and 12-15 are not directed to patent eligible subject matter.
Claim Rejections - 35 USC § 103 (AIA )
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4 and 6-15 is/are rejected under 35 U.S.C. 103 as being obvious over US 2019/392730 A1 to GUPTA in view of US 2021/201696 A1 to PEREZ.
Regarding claim 1, GUPTA teaches A system for assisting a user to learn…languages (par. 0002: a three-dimensional (3-D) simulator with real-time analysis, feedback, and remediation for public speaking practice and improvement), including: a first device for taking a picture of the user pronouncing in accordance with audio of a moving picture (FIG. 2, 3, 12a-13; par. 0070, 0076: Behavior analysis engine 154 receives a video stream of user 10 performing a presentation. The video feed is received by application 100 from camera 142; par. 0090-91: …lessons that are available include lessons on enunciation and pronunciation of words; The lessons of application 100 include instructional videos put together by professional instructors and interactive elements to guide the user through learning specific
skills critical to public speaking); a first memory storing therein exemplary non-verbal communication skills to be demonstrated by a speaker during conversation (par. 0070: Behavior analysis engine 154 looks at the behavior of user 10 while presenting the speech. Body movement, posture, gestures, facial expression, and eye contact are all analyzed. Behavior analysis engine 154 looks at body movement, gestures, and posture of user 10 to flag or output a feature if the user is fidgeting her hands, rocking back and forth, or exhibiting other undesirable body movements while presenting); a second memory storing therein a trained evaluation model for evaluating non-verbal communication skills of a speaker during conversation (par. 0080-82: Thousands of speeches 170 are input into speech analysis engine 110 to form the basis of predictive model 176. A wide variety of speeches, both good and bad, are input into the machine learning algorithm); a second device for comparing the exemplary non-verbal communication skills stored in the first memory to non-verbal communication skills of the user having been acquired by the first device, by means of the trained evaluation model stored in the second memory, to thereby evaluate non-verbal communication skills of the user (par. 0081: Each speech is input into speech analysis engine 110 to generate the same features and metrics that will be generated when user 10 uses presentation training application 100; par. 0083: Presentations of user 10 are compared against predictive model 176 to provide tips and feedback.); and a third device for displaying evaluation made by the second device (FIG. 12d, 13; par. 0100, 107-109; 114: Application 100 gives the presentation an overall score 270 in a letter grade or numerical form, and provides additional metrics, statistics, and scores). To the extent GUPTA does not expressly disclose teaching foreign languages, PEREZ teaches a related invention that includes evaluating and providing feedback to a speaker giving a presentation in a non-native foreign language (Abstract; par. 0044; 0053). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of GUPTA to provide evaluate and provide feedback to a person speaking in a non-native foreign language, as taught by PEREZ, in order to assist the user in forming the proper grammar and diction in a foreign language (par. 0044) during public speaking.
Regarding claim 2, GUPTA further teaches wherein the second device evaluates non- verbal communication skills of the user with respect to at least one items selected among countenance, gaze, gesture, body action, proxemics, physical appearance, visual focus, auditory information and cultural background (par. 0062-64: Vocalics analysis engine 150 analyzes the rhythm, intonation, and intensity of the user's voice during a presentation; par. 0070-74: Body movement, posture, gestures, facial expression, and eye contact are all analyzed).
Regarding claim 3, GUPTA further teaches wherein the second device extracts a feature degree indicating quantitatively a feature of each of the items, based on image data of the user having been acquired by the first device, and compares the thus extracted feature degree to the exemplary non-verbal communication skills stored in the first memory (par. 0079-82: Machine learning algorithm 174 receives the features and metrics from speech analysis engine 110, as well as the expert ratings 172, for each speech 170. Machine learning algorithm 174 compares the key features and metrics of each speech 170 to the ratings 172 for each speech, and outputs predictive model 176. Predictive model 176 includes rating scales for individual metric parameters and features used by application 100 to provide ratings to a presentation subsequently given by user 10. Predictive model 176 defines what features make a great speech great, and what features occur that result in a poor expert rating).
Regarding claim 4, GUPTA further teaches wherein the second device auxiliary uses conversation of the user as verbal data in evaluation of the non-verbal communication skills of the user (par. 0062-0064: Vocalics analysis engine 150 analyzes the sound generated by user 10).
Regarding claim 6, GUPTA further teaches wherein the trained evaluation model is made by machine learning so as to evaluate non-verbal communication skills of a speaker with the exemplary non-verbal communication skills stored in the first memory being used as criteria, an input to the trained evaluation model includes image data of non-verbal communication skills of the user during conversation, the image data being taken by the first device, and an output from the trained evaluation model is evaluation to the non-verbal communication skills of the user during conversation, the evaluation being made based on the exemplary non-verbal communication skills (par. 0079-82: Thousands of speeches 170 are input into speech analysis engine 110 to form the basis of predictive model 176. A wide variety of speeches, both good and bad, are input into the machine learning algorithm. Each speech is input into speech analysis engine 110 to generate the same features and metrics that will be generated when user 10 uses presentation training application 100. In addition, experts are employed to observe speeches 170 and provide ratings 172 based on the experts' individual opinions; Machine learning algorithm 174 receives the features and metrics from speech analysis engine 110, as well as the expert ratings 172, for each speech 170. Machine learning algorithm 174 compares the key features and metrics of each speech 170 to the ratings 172 for each speech, and outputs predictive model 176. Predictive model 176 includes rating scales for individual metric parameters and features used by application 100 to provide ratings to a presentation subsequently given by user 10. Predictive model 176 defines what features make a great speech great, and what features occur that result in a poor expert rating).
Regarding claim 7, GUPTA further teaches a fifth device for making curriculum specialized for the user so as to compensate for the non-verbal communication skills of the user having been low-evaluated by the second device (par. 0025, 0029, 0031, 0040, 0050, 0079: The training system observes user 10 presenting via optical, audio, and biometric inputs, and is able to give real-time feedback during the presentation, summarize performance problems after the performance, and provide tips and tutorials for improvement.).
Regarding claim 8, GUPTA further teaches a sixth device for making learning materials in line with the curriculum made by the fifth device (par. 0025, 0029, 0031, 0040, 0050, 0079: The training system observes user 10 presenting via optical, audio, and biometric inputs, and is able to give real-time feedback during the presentation, summarize performance problems after the performance, and provide tips and tutorials for improvement.).
Regarding claim 9, GUPTA further teaches wherein the learning materials include a moving picture in which characters and the user make conversation with each other (FIG. 12d, 13; par. 0103-106: application 100 renders a virtual audience including audience members 252, 254, and 256 on the screen of television 240; simulated audience includes realistic renderings of human beings, and the people in the audience react realistically to the presentation by user 10).
Regarding claim 10, GUPTA further teaches a seventh device for displaying a subtitle in the moving picture, wherein the seventh device, when non-verbal communication skills of the user having been low-evaluated by the second device appears in the conversation, displays at least a first subtitle among first and second subtitles, the first subtitle expressing low evaluation of the non-verbal communication skills of the user and a subtitle, the second subtitle including advice to the low-evaluated non-verbal communication skills of the user (FIG. 12d, ref. 262, 264; par. 0100, 107-108: notification 262 indicates when a metric is outside of a threshold goal. In FIG. 12d, user 10 is going faster than a goal set by application 100 for the user).
Regarding claim 11, GUPTA further teaches an eighth device for turning the subtitle into audio, wherein the eighth device plays the audio in the moving picture in place of or together with the subtitle (FIG. 12d, ref. 252, 254, 256; par. 0100, 107-108: The rendered crowd reacts realistically to the presentation. Application 100 also provides optional audible and on-screen alerts and status updates).
Regarding claim 12, GUPTA further teaches A portable wireless communication device including the system as set forth in claim 1 (the rejection of claim 1 is incorporated herein by reference).
Regarding claim 13, GUPTA teaches A method of assisting a user to learn …languages (par. 0002: a three-dimensional (3-D) simulator with real-time analysis, feedback, and remediation for public speaking practice and improvement), including:
taking a picture of the user pronouncing in accordance with audio of a moving picture (FIG. 2, 3, 12a-13; par. 0070, 0076: Behavior analysis engine 154 receives a video stream of user 10 performing a presentation. The video feed is received by application 100 from camera 142; par. 0090-91: …lessons that are available include lessons on enunciation and pronunciation of words; The lessons of application 100 include instructional videos put together by professional instructors and interactive elements to guide the user through learning specific
skills critical to public speaking);
comparing exemplary non-verbal communication skills to non-verbal communication skills of the user having been acquired in the picture-taking step, by means of a trained evaluation model used for evaluating non-verbal communication skills of a speaker in conversation, to thereby evaluate the non-verbal communication skills of the user, the exemplary non-verbal communication skills being read out of a memory storing therein exemplary non-verbal communication skills including exemplary countenance, gesture and so on to be demonstrated by a speaker in conversation (par. 0070: Behavior analysis engine 154 looks at the behavior of user 10 while presenting the speech. Body movement, posture, gestures, facial expression, and eye contact are all analyzed. Behavior analysis engine 154 looks at body movement, gestures, and posture of user 10 to flag or output a feature if the user is fidgeting her hands, rocking back and forth, or exhibiting other undesirable body movements while presenting; par. 0081: Each speech is input into speech analysis engine 110 to generate the same features and metrics that will be generated when user 10 uses presentation training application 100; par. 0083: Presentations of user 10 are compared against predictive model 176 to provide tips and feedback.); and
displaying evaluation made in the comparison step (FIG. 12d, 13; par. 0100, 107-109; 114: Application 100 gives the presentation an overall score 270 in a letter grade or numerical form, and provides additional metrics, statistics, and scores).
To the extent GUPTA does not expressly disclose teaching foreign languages, PEREZ teaches a related invention that includes evaluating and providing feedback to a speaker giving a presentation in a non-native foreign language (Abstract; par. 0044; 0053). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of GUPTA to provide evaluate and provide feedback to a person speaking in a non-native foreign language, as taught by PEREZ, in order to assist the user in forming the proper grammar and diction in a foreign language (par. 0044) during public speaking.
Regarding claim 14, GUPTA further teaches A recording medium readable by a computer, storing a program therein for causing a computer to carry out the method as set forth in claim 13 (the rejection of claim 13 is incorporated herein by reference).
Regarding claim 15, GUPTA further teaches A portable wireless communication device including a program for causing the portable wireless communication device to carry out the method as set forth in claim 13 (the rejection of claim 13 is incorporated herein by reference).
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being obvious over GUPTA in view of PEREZ as applied to claim 1, in further view of CA 2611053 A1 to VILHJALMSSON.
Regarding claim 5, GUPTA teaches the elements above, but does not disclose a first database storing therein cultural background data of various countries and regions; and a fourth device for reading cultural background data of the user out of the first database, and taking the cultural background data of the user into consideration in evaluation of non-verbal communication skills of the user to be carried out by the second device.
However, VILHJALMSSON teaches an interactive foreign language teaching system (Abstract) that teaches users communicative skills for interacting with people who speak foreign languages or belong to foreign cultures (par. 0040). VILHJALMSSON teaches said communicative skills may include spoken language skills in foreign languages, as well as knowledge of nonverbal communication modalities such as hand gestures and nonverbal vocalizations, as well as social norms and rules of politeness and etiquette governing conversational interaction in various settings (par. 0040). VILHJALMSSON teaches a teaching device may continually track a learner's mastery of each of a range of communicative skills, and may use this information to customize a learning experience (par. 0044). VILHJALMSSON teaches introducing cultural norms of etiquette and politeness, to help learners accomplish social interaction tasks successfully (par. 0048). VILHJALMSSON teaches cultural factors may vary from one culture to another (par. 0061). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate evaluating a user’s communication skills based on several factors, including cultural norms, as taught by VILHJALMSSON, into the modified invention of GUPTA, in order to provide more specialized feedback to the user based on their communication performance for a given culture.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to James Hull whose telephone number is 571-272-0996. The examiner can normally be reached on Monday-Friday from 8:00am to 5:00pm MST.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xuan Thai, can be reached at telephone number 571-272-7147. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/JAMES B HULL/Primary Examiner, Art Unit 3715
Prosecution Insights