Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 9/22/2026 has been entered.
Response to Amendment
The Amendment filed September 22, 2026 has been entered and considered. Claims 1-2, 8-9, 15, and 16 have been amended. Claims 4, 11, and 18 have been canceled. New claims 21-23 have been added. In light of the amendment the prior art rejections of claims 1, 8, and 15 are withdrawn as moot. The new grounds of rejection set forth in the present action were necessitated by Applicants’ claim amendments.
112(d) Rejection –
In view of the amendments to the claims, the rejection has been withdrawn as moot.
Claim Objections
Claim 2 is objected to because of the following informalities: “determining whether the derived label corresponds to the spars of singleton cluster” is recited in lines 2-3. Should read “determining whether the derived label corresponds to the sparse or singleton cluster”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-3, 5, 7-10, 12, 14-17, and 20-23 are rejected under 35 U.S.C. 103 as being unpatentable over McDorman et al. (Previously cited) in view of Koukoumidis et al. (Previously cited) further in view of Nie (US Patent Pub. No. 20200104234 A1, published 2020).
Regarding claim 1, McDorman teaches a method for identifying an event as a memorable experience associated with a user, the method comprising: receiving a data stream from each of multiple individual sensors associated with a user during an event, each data stream from each individual sensor having a raw label provided by the individual sensor, each raw label indicating what data from the data stream represents (Para. 14, “Further, the location-based telemetry application may be configured to capture sensor data from sensors of the client device that reside on the client device. The sensor data may be used to generate the location-based telemetry data. The sensor data may include, geolocation (i.e. GPS sensor), weather conditions (i.e. temperature sensor), noise pollution (i.e. microphone), user kinematic movement (i.e. accelerometers), and/or so forth.”); associating an inferred label describing aspects of the event with each individual data stream, the inferred label indicating a description of a visual scene, or a description of environmental conditions received from an external source (Para. 12, “In various examples, location-based telemetry data may relate to places of interest and/or events visited by a user over a predetermined time interval. Non-limiting examples may include geolocations visited by the client device, multimedia captured at the visited geolocations, transactions initiated via the client device at visited geolocations, weather conditions at the visited geolocations at the visited point-in-time, events taking place at the visited geolocation at the visited point-in-time, and/or any other information pertinent to a visited geolocation and/or client device”); determining, using the raw label of each of the multiple individual sensors and inferred labels of the data stream from each of the multiple individual sensors associated with the user during the event, a derived label describing an event experienced by the user (Para. 58, “The telemetry data analysis component 422 may use one or more trained machine-learning algorithms to analyze the location-based telemetry data and infer a context associated with the visit. For example, the telemetry data may include a geolocation from a GPS sensor of the client device along with calendar data from a third-party calendar application that resides on the client device. In this example, the telemetry data analysis component 422 may infer that the context of visit relates to a schedule appointment.”); based at least in part on the inferred and derived label, determining, whether the event experienced by the user is a memorable event for the user (Para. 11, “The location-based telemetry data is intended to capture information about a user, that is specific to the user, but at the same time, not traditionally known or captured as part of a user profile.”), and based at least in part on determining that the event experienced by the user is a memorable event, utilizing the memorable event as a biometric authentication factor in a multi-factor authentication process (Para. 16, “By way of example, an authentication challenge may ask the user to identify a third-party with whom they conducted a voice communication (i.e. phone call) at a geolocation at a particular point in time, an event or landmark visited at the geolocation a particular point in time, or a weather condition or noise pollution experienced at the geolocation a particular point in time.”).
McDorman does not explicitly disclose indicating whether a positive or negative reaction is experienced by the user, wherein an inferred label indicating a positive reaction qualifies as a memorable event and an inferred label indicating a negative reaction does not qualify as a memorable event, or mapping the derived label to a vector space comprising a plurality of derived labels describing other events previously experienced by the user and determining whether the derived label corresponds to a sparse or singleton cluster within the vector space. However, they disclose examples of data related to places of interest, specific events, user kinematics, or multimedia captured at the visited location, as well as disclosing the possibility of using machine-learning algorithms. With respect to the present specification, [0029-0030] describe certain labels as “inferred” and certain labels as “derived”, where the derived labels are a combination of the (orthogonal/uncorrelated) inferred labels and are not necessarily directly measuring the mood of the user (i.e. the skateboarder and teddy bear examples).
Koukoumidis teaches indicating whether a positive or negative reaction is experienced by the user (Para. 26, “Alternatively or additionally, these devices may implement machine learned emotional signals from supervised labels where human annotators label data as identifying an emotion or some other sentiment (e.g., a positive or negative sentiment).”) and wherein an inferred label indicating a positive or negative reaction qualifies as a memorable event (Para. 17, “As described herein, users experience numerous events every day, some of which are memorable events that carry positive or negative emotional weight.”).
Koukoumidis does not explicitly disclose that an inferred label indicating a negative event does not qualify as a memorable event, that an inferred label indicating a positive reaction is a memorable event, or mapping the derived label to a vector space comprising a plurality of derived labels describing other events previously experienced by the user and determining whether the derived label corresponds to a sparse or singleton cluster within the vector space. However, they do acknowledge that memorable events sort into two known categories, positive and negative, which both have a distinct purpose (Para. 17). They also disclose the possibility of using machine learning algorithms for analysis.
Nie teaches mapping the derived label to a vector space comprising a plurality of derived labels describing other events previously experienced and determining (Para. 10, “By way of further example, a model employed by an embodiment may map words (representing events) to vectors in a lower-dimension space.”), whether the derived label corresponds to a sparse or singleton cluster within the vector space (Para. 16, “In an embodiment, clustering the identified repeating events into one or more groups may comprise: clustering the identified repeating events into a plurality of groups; analyzing the plurality of groups to identify an outlier to the plurality of groups…”; Para. 17, “Also, analyzing the plurality of groups to identify an outlier to the plurality of groups may comprise: comparing the number of members of a group with a threshold value; and identifying the group as an outlier based on comparison result.”).
Nie does not explicitly disclose that the events are previously experienced specifically by a user. However, they do consider previous events.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified McDorman to incorporate the teachings of Koukoumidis and Nie to include indicating whether a positive or negative reaction is experienced by the user and wherein an inferred label indicating a positive reaction qualifies as a memorable event and an inferred label indicating a negative reaction does not qualify as a memorable event, and mapping the derived label to a vector space comprising a plurality of derived labels describing other events previously experienced by the user and determining, whether the derived label corresponds to a sparse or singleton cluster within the vector space. McDorman discloses a system which provides an authentication challenge by leveraging user location and event specific data in order to provide a challenge which is specific to the user but not traditionally known. They do not explicitly disclose utilizing inferred emotional reaction information for the authentication challenge or mapping events to a vector for clustering. Koukoumidis discloses a method of identifying an emotional reaction experienced by a user in order to leverage the memorable event for later engagement. One of ordinary skill in the art would have understood that an intense emotional reaction will cause an event to be more memorable, as disclosed by Koukoumidis, providing a clear motivation to use memorable events in the authentication challenge of McDorman, whose goal is to provide challenges that only a specific user can recall. Koukoumidis also recognizes that memorable events sort into two known categories, positive or negative. When using these memorable events in an authentication system, selecting which of these two already identified categories to draw challenges from is a choice between a predictable set of options (positive, negative, or both). One of ordinary skill in the art would have recognized that prompting a user with a recalled negative or traumatic event during a routine login flow is undesirable, leaving it obvious to restrict the challenge pool to positive events when adapting the generalized memorable-event classifier of Koukoumidis to McDorman’s specific use case of a security challenge. Further, Nie discloses an event log outlier detection system, which maps events to a vector space for clustering and then identification of an outlier cluster (singleton or sparse based on the threshold and specific clusters being analyzed). In view of the disclosure of non-limiting machine learning algorithms possibly being used in both McDorman and Koukoumidis, and McDorman’s method of event based authentication which is specific to a user, one of ordinary skill in the art would have recognized that performing vector mapping of numerous events into a cluster analysis looking for singleton or sparse clusters of events, as disclosed by Nie, would have predictably improved the memorable event selection for authentication in the modified system of McDorman and Koukoumidis.
Regarding claim 2, McDorman as modified above teaches all of the elements of claim 1, as stated above, as well as wherein determining whether the derived label corresponds to the sparse or singleton cluster event experienced is memorable further comprises: determining whether the event corresponds to one or more events having previously been experienced by the user during a predetermined window of time represented in the vector space (Para. 11, “For example, a user may frequent a merchant store (i.e. coffee store) on particular days of the week and at particular times of the day”; Nie; Para. 18, “For instance, the model may be generated based on previous executions of the process. Such use of historical or learned information relating to the executer process may improve an accuracy of the obtained model”); and based on the event not corresponding to one or more events having previously been experienced by the user during the predetermined window of time, determining that the derived label corresponds to the sparse or singleton cluster, the event is a memorable event (Para. 38, “Alternately, the location-based telemetry application may forgo a prompt to the client to select the set of geolocations, and instead unobtrusively generate the set of authentication challenges based on geolocations that are frequently, and/or infrequently visited.”; Para. 37, “The benefit of doing so is that the client may add an additional level of complexity to the authentication challenges”; Para. 59, “Complexity may increase with authentication challenges that are based on geolocations that are infrequently visited by a client.”; Increased complexity is beneficial as it makes the authentication challenge more difficult for a third-party malicious actor, and infrequent locations are disclosed as increasing complexity, leaving it obvious to determine an infrequent event as a memorable event; Nie; Para. 16, “In an embodiment, clustering the identified repeating events into one or more groups may comprise: clustering the identified repeating events into a plurality of groups; analyzing the plurality of groups to identify an outlier to the plurality of groups…”; Para. 17, “Also, analyzing the plurality of groups to identify an outlier to the plurality of groups may comprise: comparing the number of members of a group with a threshold value; and identifying the group as an outlier based on comparison result.”).
Regarding claim 3, McDorman as modified above teaches all of the elements of claim 1, as stated above, as well as wherein the receiving, the associating, the determining a derived label, and the determining whether the event experienced by the user is a memorable event for the user are executed by a neural network trained to identify memorable events (Para. 58, “The telemetry data analysis component 422 may use one or more trained machine-learning algorithms to analyze the location-based telemetry data and infer a context associated with the visit”; Para. 76, “The one or more machine learning algorithms may include but are not limited to algorithms such as… neural networks”; Koukoumidis; Para. 26, “Alternatively or additionally, these devices may implement machine learned emotional signals from supervised labels where human annotators label data as identifying an emotion or some other sentiment (e.g., a positive or negative sentiment).”).
Regarding claim 5, McDorman as modified above teaches all of the elements of claim 1, as stated above, as well as wherein determining the derived label describing the event experienced by the user further comprises using data associated with the user received from one or more external sources associated with the user (Para. 14, “The sensor data may be used to generate the location-based telemetry data. The sensor data may include, geolocation (i.e. GPS sensor), weather conditions (i.e. temperature sensor), noise pollution (i.e. microphone), user kinematic movement (i.e. accelerometers), and/or so forth.”).
Regarding claim 7, McDorman as modified above teaches all of the elements of claim 1, as stated above, as well as wherein the multiple individual sensors associated with the user comprise sensors for tracking biomarkers and vital signs associated with the user (Koukoumidis; Para. 23, “Emotional reaction information may be in the form of raw data (e.g., biometric information, speech, text, video feed, etc.).”; Para. 25, “In an example, the one or more detection devices may detect a user's emotional reaction information by measuring the user's biometric information such as the user's heart rate, heart rate variability, breathing pattern, amount of perspiration using a galvanic skin response device, etc.”).
Claim 8 corresponds to claim 1 and is rejected under the same analysis.
Claim 9 corresponds to claim 2 and is rejected under the same analysis.
Claim 10 corresponds to claim 3 and is rejected under the same analysis.
Claim 12 corresponds to claim 5 and is rejected under the same analysis.
Claim 14 corresponds to claim 7 and is rejected under the same analysis.
Claim 15 corresponds to claim 1 and is rejected under the same analysis.
Claim 16 corresponds to claim 2 and is rejected under the same analysis.
Claim 17 corresponds to claim 3 and is rejected under the same analysis.
Claim 20 corresponds to claim 7 and is rejected under the same analysis.
Regarding claims 21-23, the recited elements perform variably the same function as the recited elements of claim 1. Specifically, the limitation of claim 1 stating “an inferred label indicating a negative reaction does not qualify as a memorable event”, and is rejected under the same analysis.
Claim(s) 6, 13, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over McDorman as modified in view of Koukoumidis and Nie, further in view of Etkin (US Patent Pub. No. 2014/0040653 A1, previously cited).
Regarding claim 6, McDorman as modified in view of Koukoumidis and Nie teaches all of the elements of claim 1, as stated above, as well as using multiple sensors.
They do not explicitly disclose performing interpolation on one or more individual data streams to fill in gaps in the individual data streams when an individual sensor associated with collecting the individual data stream has a sampling rate that is lower than one or more other individual sensors.
Etkin teaches performing interpolation on one or more individual data streams to fill in gaps in the individual data stream when an individual sensor associated with collecting the individual data stream has a sampling rate that is lower than one or more other individual sensors of the multiple individual sensors (Figs. 1, 10, Para. 20, “The method includes receiving a sequence of time-stamped data indicative of physical events from each of a plurality of sensors (100), generating an interpolation filter according to desired sampling times (102), and interpolating the sequences of time-stamped data with the generated filter to obtain sequences of data synchronized at desired sampling times (104).”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified McDorman, Koukoumidis, and Nie to incorporate the teachings of Etkin to include performing interpolation on one or more individual data streams to fill in gaps in the individual data stream when an individual sensor associated with collecting the individual data stream has a sampling rate that is lower than one or more other individual sensors of the multiple individual sensors. McDorman teaches using multiple different sensors to capture data associated with a user at an event, however they do not mention synchronizing the data captured by these different sensors, leaving possible gaps in the captured data. Etkin teaches to perform interpolation to obtain sequences of data synchronized at desired sampling times. One of ordinary skill in the art would understand that implementing the interpolation techniques of Etkin into the method of McDorman would provide the predictable benefit of more robust sensor data.
Claim 13 corresponds to claim 6 and is rejected under the same analysis.
Claim 19 corresponds to claim 6 and is rejected under the same analysis.
Conclusion
Pertinent Prior Art: Steinbach, M. (2000, October 2). An introduction to cluster analysis for data mining. https://civil.colorado.edu/~balajir/CVEN6833/lectures/cluster_survey_10_02_00.pdf Shows that cluster analysis is a well-known technique in the art. Includes descriptions of singleton clusters as well as the benefits of sparsification.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID A WAMBST whose telephone number is (703)756-1750. The examiner can normally be reached M-F 9-6:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ALEXANDER WAMBST/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
