DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments and amendments in the amendment filed March 12, 2026 (herein “Amendment”), with respect to the invocation of interpretation of 35 U.S.C. 112(f) for claim 10, and the subsequent rejection under 35 U.S.C. 112(b) for claim 10 have been fully considered and are persuasive. Claim 10 is no longer interpreted as invoking 35 U.S.C. 112(f) and the rejection of claim 10 under 35 U.S.C. 112(b) has been withdrawn.
Applicant’s arguments and amendments in the Amendment, with respect to the rejection of claims 1–11 (now only 1 and 10–11 left pending) under 35 U.S.C. 101 for being directed towards an abstract idea without a practical application or significantly more have been fully considered and are persuasive. The rejection of remaining pending claims 1 and 10–11 under 35 U.S.C. 101 has been withdrawn.
Applicant's arguments and amendments in the Amendment regarding the constructive rejection of claims 1 and 10–11 under 35 U.S.C. 103 in view of Danilo and Rodriguez have been fully considered but they are not persuasive. Specifically, independent claims 1, 10 and 11 have been amended to include the limitations of previously pending claims 2–8 which, at least for claims 4–6, relied upon the combination of both Danilo and Rodriguez. Applicant’s remarks regarding primary reference Danilo not teaching or suggesting the newly amended “and wherein the correlation analysis is performed through canonical correlation analysis (CCA); classifying the personality expression space by decision boundaries;” and “predicting a personality … including determining a mapped point on the personality expression space by the mapping and predicting the personality based on a class determined by the decision boundaries for the mapped point,” are not responsive to the constructive rejection as these limitations were previously recited in claims 4–6 and for these limitations, Rodriguez was relied upon. In the Amendment’s remarks, Applicant does not argue against the relied upon teachings of Rodriguez; rather arguments are set forth on pages 11–12 regarding the rationale to combine Rodriguez with Danilo. Here Applicant appears to argue that there would be no reason to modify Danilo with Rodriguez’s correlation analysis through CCA and mapping on a personality expression because Danilo already performs personality assessment with its own prediction mechanisms. However, as given in the Non-Final Office Action issued December 16, 2025, on page 15, such a combination is motivated in that the strongest insights (strongest accuracy) is realized by the included teachings of Rodriguez. Applicant does not contest that motivation of record in their remarks, and the Examiner finds that this motivation rationale is proper. Further, in response to Applicant's argument that the examiner's conclusion of obviousness is based upon improper hindsight reasoning, it must be recognized that any judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight reasoning. But so long as it takes into account only knowledge which was within the level of ordinary skill at the time the claimed invention was made, and does not include knowledge gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971).
Therefore, in view of the above, while all of Applicant’s arguments have been fully considered, they are not persuasive, and the constructive rejection against claims 1 and 10–11 is maintained.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 10–11, 14 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Danilo et al., US Patent Application Publication No. US 2021/0050000 A1 (herein “Danilo”), further in view of Rodriguez et al., “Structural Correlates of Personality Dimensions in Healthy Aging and MCI,” Front. Psychol., January 7, 2019, https://doi.org/10.3389/fpsyg.2018.02652 (herein “Rodriguez”).
Regarding claims 1 and 10, with claim 1 as exemplary, and substantive differences between the claims noted in curly brackets {}, and deficiencies of Danillo noted in square brackets [], Danilo teaches a personality prediction {method / system} comprising (Danilo ¶ 31, fig. 1, systems and methods for generating a personality assessment for a user):
{at least one processor configured to: - claim 10 (Danillo ¶100, modules and circuits disclosed therein implemented with a general purpose processor)]
extracting, {by at least one processor – claim 1(Danillo ¶100, modules and circuits disclosed therein implemented with a general purpose processor)}, multimodal information from an input image in which a user appears (Danilo ¶¶ 54–56, personality data comprised of video and audio files (multimodal) are processed by a feature extraction component to extract parts of speech feature data, emotion features data, linguistic inquiry word count features, audio features and video features), wherein the multimodal information includes visual information, voice information, and text information (Danilo ¶ 55, feature extraction component extracts features from the audio files associated with the user personality video, as well as video features, and where the audio features are further analyzed to determine a text transcript from the audio), and wherein the text information includes utterance text and caption information (Danilo ¶ 55, audio files from the video are analyzed to determine the text transcript (utterance text and caption information));
extracting features from the extracted multimodal information (Danilo ¶ 55, feature extraction including extracting audio features of spoken words and video features from a user personality video (including at least one input image since video is a sequence of images) of the user);
generating a multimodal feature by merging the extracted features (Danilo ¶¶ 59–60, a correlation coefficient algorithm is used to combine (merging) features from the different modalities of video and audio by deleting features highly correlated with another feature, and then reduced dimensionality dataset are annotated so that data from one modality that is associated with one of the big 5 personality traits such as openness is integrated with POS (part of speech – different modality) data from the same openness personality trait);
mapping the multimodal feature on a personality expression space (Danilo ¶¶ 52, 61, 85, 88–89, feature data which is multimodal, is processed by classification prediction process involving obtaining (mapping) to a big 5 classification prediction value in 5 dimensions (personality expression space) for each of the feature types) that is constituted by a plurality of personality indexes (Danilo ¶ 52, big 5 personality traits as the factors in the five-factor model for personality assessment, the five factor model being a dimensional classification model of general personality structure that rates the subject on five dimensions of personality features (personality indices) including openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism), wherein the mapping is performed based on a result of analyzing a correlation between the multimodal feature and corresponding personality indexes (Danilo ¶¶ 52, 60, the five factor model is multimodal, and the features of the multimodal data are associated with (correlation) the personality traits from the five-factor model (personality indexes)), [and wherein the correlation analysis is performed through canonical correlation analysis (CCA)];
[classifying the personality expression space by decision boundaries;] and
predicting a personality of the user based on a result of the mapping (Danilo ¶¶ 89, 94, final big 5 personality traits are calculated from the big 5 classification prediction values calculated for each of the feature types, and output as a Final Prediction including a Personality Predictor to the client) [including determining a mapped point on the personality expression space by the mapping and predicting the personality based on a class determined by the decision boundaries for the mapped point].
Danilo does not explicitly teach, but Rodriguez teaches wherein the correlation analysis is performed through canonical correlation analysis (CCA) (Rodriguez page 4, Personality Factors and TBSS subsection, Canonical Correlation Analysis is applied separately for each personality dimension);
classifying the personality expression space by decision boundaries (Rodriguez page 7, fig. 3, best fit line defining a boundary for the points of a scatter plot for various personality dimensions (personality expression space) such as agreeableness, conscientiousness, and openness);
including determining a mapped point on the personality expression space by the mapping and predicting the personality based on a class determined by the decision boundaries for the mapped point (Danilo page 7, fig. 3, dots (mapped point) corresponding to subjects mapped on the space defined by an x axis Canonical value of TBSS, and y axis of the canonical variate of personality, predict the personality dimensions of Agreeableness, Conscientiousness and Openness).
Therefore, taking the teachings of Danilo and Rodriguez together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have the personality expression space, and correlation analysis in Danilo to be classified by boundaries and groupings of datapoints, and using CCA as disclosed in Rodriguez at least because doing so would allow for determining maximal correlation between canonical variables, thus producing the strongest insights of correspondence between two different sets of data. See Rodriguez page 6, fig. 2 caption.
Regarding claim 11, with deficiencies of Danilo noted in square brackets [], Danilo teaches a processor-implemented personality prediction method comprising (Danilo ¶ 31, fig. 1, method for generating a personality assessment for a user, where ¶100 teaches modules and circuits disclosed therein implemented with a general purpose processor):
mapping a multimodal feature extracted from an input image in which a user appears on a personality expression space (Danilo ¶¶ 52, 54–56, personality data comprised of video and audio files (multimodal) are processed by a feature extraction component to extract parts of speech feature data, emotion features data, linguistic inquiry word count features, audio features and video features, the features being correlated to a big 5 classification prediction value in 5 dimensions (personality expression space) for each of the feature types) which is constituted by a plurality of personality indexes (Danilo ¶ 52, big 5 personality traits as the factors in the five-factor model for personality assessment, the five factor model being a dimensional classification model of general personality structure that rates the subject on five dimensions of personality features (personality indices) including openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism), wherein the mapping is performed based on a result of analyzing a correlation between the multimodal feature and corresponding personality indexes (Danilo ¶¶ 52, 60, the five factor model is multimodal, and the features of the multimodal data are associated with (correlation) the personality traits from the five-factor model (personality indexes)), [and wherein the correlation analysis is performed through canonical correlation analysis (CCA)];
[classifying the personality expression space by decision boundaries;] and
predicting a personality of the user based on a result of the mapping (Danilo ¶¶ 89, 94, final big 5 personality traits are calculated from the big 5 classification prediction values calculated for each of the feature types, and output as a Final Prediction including a Personality Predictor to the client), [including determining a mapped point on the personality expression space by the mapping and predicting the personality based on a class determined by the decision boundaries for the mapped point].
Danilo does not explicitly teach, but Rodriguez teaches wherein the correlation analysis is performed through canonical correlation analysis (CCA) (Rodriguez page 4, Personality Factors and TBSS subsection, Canonical Correlation Analysis is applied separately for each personality dimension);
classifying the personality expression space by decision boundaries (Rodriguez page 7, fig. 3, best fit line defining a boundary for the points of a scatter plot for various personality dimensions (personality expression space) such as agreeableness, conscientiousness, and openness);
including determining a mapped point on the personality expression space by the mapping and predicting the personality based on a class determined by the decision boundaries for the mapped point (Danilo page 7, fig. 3, dots (mapped point) corresponding to subjects mapped on the space defined by an x axis Canonical value of TBSS, and y axis of the canonical variate of personality, predict the personality dimensions of Agreeableness, Conscientiousness and Openness).
Therefore, taking the teachings of Danilo and Rodriguez together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have the personality expression space, and correlation analysis in Danilo to be classified by boundaries and groupings of datapoints, and using CCA as disclosed in Rodriguez at least because doing so would allow for determining maximal correlation between canonical variables, thus producing the strongest insights of correspondence between two different sets of data. See Rodriguez page 6, fig. 2 caption.
Regarding claims 14 and 23, with claim 14 as exemplary, Danilo teaches wherein the plurality of personality indexes utilize Big Five factors (OCEAN), including an openness to experience index (O), a conscientiousness index (C), an extraversion index (E), an agreeableness index (A), and a neuroticism index (N) (Danilo ¶¶8, 88, and 95, fig. 6, reference number 603, personality assessment based on the big 5 personality traits of openness to experience, conscientiousness, extraversion, agreeableness and neuroticism).
Claims 12–13, 16–19, 21–22, and 25–28 are rejected under 35 U.S.C. 103 as being unpatentable over Danilo, further in view of Rodriguez, as set forth above regarding claims 1 and 10 from which claims 12–13, 16–19, 21–22, and 25–28 respectively depend, further in view of Shin et al., US Patent No. 12,541,997 B2 (herein “Shin”).
Regarding claims 12 and 21, with claim 12 as exemplary , Danilo teaches wherein the extracting of the multimodal information comprises: generating the utterance text by converting an uttered voice of the user through Speech-to-Text (STT) (Danilo ¶55, audio files associated with the video are analyzed and a speech-to-text application is used to generate a text transcript).
Danilo does not explicitly teach where Shin teaches generating the caption information indicating a facial expression of the user or a situation derived from the input image through video captioning (Shin col. 19, ll. 4–28, and col. 13, ll. 40–51, caption generation unit generates a caption for each scene (indicating a situation) on the basis of an impression identification unit of a captured content, where the impression identification unit identifies a facial expression which is used to generate a label of the impressing scene (indicating a facial expression)); and extracting features of the voice information using a Mel-frequency cepstral coefficient (MFCC) algorithm (Shin col. 19, ll. 15–18, audio data in the video is converted into a feature such as MFCC).
Therefore, taking the teachings of Danilo as modified by Rodriguez and Shin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the correlation analysis in Danilo to include captioning and MFCC feature analysis as disclosed in Shin at least because doing so would allow for efficient observation prediction from multimodal data. See Shin col. 30, ll. 33–49 and Abstract.
Regarding claims 13 and 22, where claim 13 is exemplary, Danilo does not explicitly teach, but Shin teaches wherein the generating of the multimodal feature comprises: averaging features of the visual information, features of the voice information, features of the utterance text, and features of the caption information, respectively (Shin fig. 4, col. 14, ll. 34–40, col. 14, l. 54–col. 15, l. 7, and col. 19, ll. 22–28, multi modal data is collected and processed in averaged units of segments, including a series of frames 1-n for the generated video feature (visual information), the audio data synchronized with the video over the segment, and the text data including the caption information determined from the audio data); and converting each of the averaged features into one vector element to form the multimodal feature as a single feature vector (Shin fig. 8, col. 19, ll. 22–35, the video feature, (features of the visual information), audio feature (features of the voice information), and text feature which includes superimposed caption information (features of utterance text and features of the caption information) are projected into a common space (converted into one vector element as a single feature vector in the impression identification space) to result in an impression identification).
Therefore, taking the teachings of Danilo as modified by Rodriguez and Shin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the correlation analysis in Danilo to include the feature analysis as disclosed in Shin at least because doing so would allow for efficient observation prediction from multimodal data. See Shin col. 30, ll. 33–49 and Abstract.
Regarding claims 16 and 25, with claim 16 as exemplary, Danilo does not explicitly teach, but Shin teaches wherein the extracting of the multimodal information comprises extracting the multimodal information from a plurality of input images (Shin fig. 4, col. 14, ll. 36–42, 54–57, multiple frames of the video (plurality of input images) are input into the network unit to determine a features for each, and where the audio data and text transcript thereof are data synched with the video), and wherein the extracting of the features and the generating of the multimodal feature are repeated with respect to the plurality of input images (Shin fig. 4, col. 14, ll. 54–57, features for a video feature are obtained for each frame with a CNN mode respective to the frame (repeated)).
Therefore, taking the teachings of Danilo as modified by Rodriguez and Shin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the correlation analysis in Danilo to include the feature analysis as disclosed in Shin at least because doing so would allow for efficient observation prediction from multimodal data. See Shin col. 30, ll. 33–49 and Abstract.
Regarding claims 17 and 26, with claim 17 as exemplary, Danilo as modified by Rodriguez does not explicitly teach, but Shin teaches wherein the extracting of the features comprises extracting visual information features, extracting voice information features, and extracting text information features from the extracted multimodal information (Shin fig. 8, col. 19, ll. 5–35, and col. 14, l. 54–col. 15, l. 15, the video feature, (visual information features), audio feature (voice information features), and text feature which includes superimposed caption information (text information features) are extracted from the video frames and synchronized audio and projected into a common space).
Therefore, taking the teachings of Danilo as modified by Rodriguez and Shin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the correlation analysis in Danilo to include the feature analysis as disclosed in Shin at least because doing so would allow for efficient observation prediction from multimodal data. See Shin col. 30, ll. 33–49 and Abstract.
Regarding claims 18 and 27, with claim 18 as exemplary, Danilo as modified by Rodriguez does not explicitly teach, but Shin teaches wherein the text information includes caption information through video captioning (Shin fig. 8, col. 19, ll. 22–28, the text data including the caption information determined from the audio data which is synched with the video).
Therefore, taking the teachings of Danilo as modified by Rodriguez and Shin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the correlation analysis in Danilo to include the caption information as disclosed in Shin at least because doing so would allow for efficient observation prediction from multimodal data. See Shin col. 30, ll. 33–49 and Abstract.
Regarding claims 19 and 28, with claim 18 as exemplary, Danilo as modified by Rodriguez does not explicitly teach, but Shin teaches wherein the generating of the multimodal feature comprises deriving a vector in which modality information is merged (Shin fig. 8, col. 19, ll. 22–35, the video feature, (features of the visual information), audio feature (features of the voice information), and text feature which includes superimposed caption information (features of utterance text and features of the caption information) are projected into a common space (merged) to result in an impression identification (vector)).
Therefore, taking the teachings of Danilo as modified by Rodriguez and Shin together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the correlation analysis in Danilo to include the feature analysis as disclosed in Shin at least because doing so would allow for efficient observation prediction from multimodal data. See Shin col. 30, ll. 33–49 and Abstract.
Claims 15 and 24, are rejected under 35 U.S.C. 103 as being unpatentable over Danilo, further in view of Rodriguez, as set forth above regarding claims 1 and 10 from which claims 15 and 24 respectively depend, further in view of Ghosh et al., "A Study on Support Vector Machine based Linear and Non-Linear Pattern Classification," 2019 International Conference on Intelligent Sustainable Systems (ICISS), Palladam, India, 2019, pp. 24-28, doi: 10.1109/ISS1.2019.8908018 (herein “Ghosh”).
Regarding claims 15 and 24, with claim 15 as exemplary, Danilo teaches learning images labeled with the personality indexes (Danilo ¶60, video annotations (images labeled) for agreeableness, extraversion and neuroticism integrated into annotated features data, where ¶56 teaches features data used for training by a training component (learning)), but does not explicitly teach where Ghosh teaches wherein the decision boundaries are generated using a Support Vector Machine (SVM) based on a supervised learning using learning images labeled with the personality indexes (Ghosh page 24, section I, a supervised learning algorithm for SVM to analyze (generate) the decision boundaries).
Therefore, taking the teachings of Danilo as modified by Rodriguez and Ghosh together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the correlation analysis in Danilo to include the supervised SVM decision boundaries as disclosed in Ghosh at least because doing so would allow for optimized and accurate results in the domain. See Ghosh page 27, section V.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Danilo, further in view of Rodriguez, as set forth above regarding claim 1 from which claim 20 depends, further in view of Gite et al., "Evaluating facial expressions in real time," 2017 Intelligent Systems Conference (IntelliSys), London, UK, 2017, pp. 849-855, doi: 10.1109/IntelliSys.2017.8324228. (herein “Gite”).
Regarding claim 20, Danio as modified by Rodriguez teaches a personality label (Danio ¶61, personality assessment for an input, where ¶8 teaches the personality labels including one of the big 5 OCEAN personality features) but does not explicitly teach, where Gite teaches further comprising receiving a learning image and a label, and wherein the classifying of the personality expression space by the decision boundaries comprises generating the decision boundaries based on the learning image and the label (Gite page 854, the CK+ dataset for training including images of faces and the associated emotion label, where page 855, section V teaches a Support Vector Machine used to classify the expressions, and where page 852 section C teaches that Support Vector Machines in performing expression classification is a supervised learning algorithm (receiving a learning image and a label) and classifies by establishing decision boundary by way of a hyperplane (fig. 8)).
Therefore, taking the teachings of Danilo as modified by Rodriguez and Gite together as a whole, it would have been obvious to a PHOSITA before the effective filing date of the claimed invention to have modified the correlation analysis in Danilo to include the supervised SVM decision boundaries using a dataset with labeled images as disclosed in Gite at least because doing so would allow for greater speed while maintaining high accuracy in detecting emotion from images, where emotion is also useful for behavior and personality detection as they are correlated/linked. See Ghosh page 853, section D(3), Abstract and page 855 section V.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Kollada et al., US Patent Application Publication No. US 2022/0392637 A1, directed towards determining a mental health condition by calculating multi-modal embedding vectors from multimodal sensor data.
Applicant's amendment necessitated any new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHELLE M KOETH whose telephone number is (571)272-5908. The examiner can normally be reached Monday-Thursday, 09:00-17:00, Friday 09:00-13:00, EDT/EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
MICHELLE M. KOETH
Primary Examiner
Art Unit 2671
/MICHELLE M KOETH/Primary Examiner, Art Unit 2671
Prosecution Insights