DETAILED ACTION
The following claims are pending in this office action: 1-20
Claims 1, 17 and 20 are independent claims.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings filed on 05/05/2025 are accepted.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/05/2025 has been considered. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, an initialed and dated copy of Applicant’s IDS form 1449 filed 05/05/2025 is attached to the instant Office action.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-20 recites the limitation “the data” (claim 1, ln. 9; claim 2, ln. 1; claim 6, ln. 1; claim 9, ln. 2; claim 10, ln. 3; claim 11, ln. 9; claim 15, ln. 1; claim 17, ln. 14; claim 18, ln. 1; and claim 20, ln. 10). There is insufficient antecedent basis for this limitation in the claim as it is unclear which “data” is the intended referenced limitation. See, for example, claim 1, ln. 4: “sensor data” and claim 1, ln. 7 “embedding data.” Examiner suggests replacing “the data” with “the sensor data”, “the embedded data”, or another data as appropriate.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-10 and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Frank et al. (US Pub. 20220/0397306) (hereinafter “Frank”) corresponding to US Patent No. 11,154,203 cited in the IDS in view of Chao et al. (US Pub. 2022/0160311) (hereinafter “Chao”).
As per claim 1, Frank teaches a method comprising: ([Frank, para. 0015] Described herein are embodiments of ... methods that utilize images of a user’s face”)
at a wearable electronic device having a processor: ([Frank, para. 0027] “Embodiments ... implemented using various head-mounted objects that are worn on the user's head ... includes a helmet or goggles configured to be worn on the user's head”; [para. 0089] “Examples of electronics that may be included in smartglasses and/or a smart-helmet include ... a processor”)
generating a video signal depicting a current appearance of a face portion, ([Frank, para. 0114] “first and second head-mounted inward-facing cameras 332 that take images 333 [generating a video signal] of respective first and second regions on a user's face [of a face portion]”; [para. 0135] “The current images [a current appearance of a face portion] are taken”) the video signal generated based on sensor data captured via one or more sensors, ([para. 0089] “a camera housing that is physically coupled to a helmet, where the housing includes the imaging sensor”; [para. 0126] “each of these cameras is a multi-pixel video camera having a CMOS or a CCD sensor ... capture images [the video signal] at various rates [based on sensor data]”) wherein changes in an attribute of the face portion in the video signal over time correspond to a current heart rate of a user wearing the wearable electronic device. ([Para. 0061] “a non-contact camera that captures images of the skin, where a computer extracts the PPG signal [attribute of the face portion – see para. 0171: “one or more feature values indicative of the user’s heart rate ... derived from images ... by performing calculations on iPPG signals extracted from the images”] from the images [in the video signal] using an imaging photoplethysmography (iPPG) technique”; [para. 0183] “the computer 340 calculates, based on the current set of images, a current heart rate”; [para. 0320] “a time series signal [changes over time] is generated from video images of a subject’s exposed skin ... to obtain the PPG signal, which may be analyzed to determine ... heart rate”; the heart rate is of a user wearing the electronic device: see for example, Fig. 2b where the user wearing the electronic device is depicted)
Frank does not clearly teach embedding data into the video signal by altering the attribute of the face portion in the video signal over time such that the changes in the attribute of the face portion in the video signal over time correspond to both the current heart rate and the data; and presenting the video signal depicting the current appearance of the face portion on an outward-facing display of the wearable electronic device.
However, Chao teaches embedding data into the video signal ([Chao, para. 0015] “frontend 102 may access video data from a camera [the video signal] ... the user interface of frontend 102 may ... display determination result 106 to a user”; [Fig. 1; para. 0018] “determination result 106 may include ... one or more of an image 106a captured by the camera, [video signal] a face detection result 106b, a textual indication 106c of the determined heart rate, and a graphical indication 106d of the heart rate [data embedded into the video signal]”) by altering the attribute of the face portion in the video signal over time ([para. 0019] “To extract PPG signals ... one or more faces may be detected in the videos... detector's super-realtime performance [over time – see also para. 0013 describing the monitoring as continuous/over time] ... enable it to quickly locate an accurate facial region of interest (ROI) for PPG signal extraction [altering the attribute/PPG signal of the face portion in the video]”) such that the changes in the attribute of the face portion in the video signal over time correspond to both the current heart rate ([para. 0019] “record and/or process PPG signals, and may calculate heart rates based on the PPG signals”) and the data; and ([para. 0019] “determination result 106 ... include ... a textual indication 106c of the determined heart rate, and a graphical indication 106d of the heart rate”; as the changes in the attribute/PPG signal of the face over time are used to calculate the current heart rate, and the current heart rate is the data embedded into the video signals, the changes in the attribute necessarily correspond to “the data”)
presenting the video signal depicting the current appearance of the face portion ([Chao, para. 0015] “the user interface of frontend 102 may provide or display determination result 106 to a user [provide the current appearance of the face portion]”) on an outward-facing display of the wearable electronic device. ([Para. 0024] “user device 210 may perform one or more functions associated with frontend 102”; [para. 0025] “User device 210 includes ... a wearable device”; [para. 0040] the device includes a display that provides “output [outward facing] information”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to have modified the elements disclosed by Frank with the teachings of Chao to include embedding data into the video signal by altering the attribute of the face portion in the video signal over time such that the changes in the attribute of the face portion in the video signal over time correspond to both the current heart rate and the data; and presenting the video signal depicting the current appearance of the face portion on an outward-facing display of the wearable electronic device. One of ordinary skill in the art would have been motivated to make this modification because such an application beneficially allows for monitoring of the graph and face in everyday scenarios like working in front of a computer. (Chao, para. 0011)
As per claim 2, Frank in view of Chao teaches claim 1.
Frank does not clearly teach wherein the data is a numerical code.
However, Chao teaches also teaches wherein the data is a numerical code. ([Chao, Fig. 1; para. 0018] “determination result ... include ... a textual indication 106c of the determined heart rate [a numerical code as the heart rate is a numerical code of a number of times the heart beats per unit of time]”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to have modified the elements disclosed by Frank with the teachings of Chao to include wherein the data is a numerical code. One of ordinary skill in the art would have been motivated to make this modification because this method of output reveals indications for health-related conditions which otherwise may not be noticeable. (Chao, para. 0011)
As per claim 3, Frank in view of Chao teaches claim 1.
Frank also teaches wherein the wearable electronic device is a head mounted device (HMD) ([Frank, para. 0015] “the images ... captured using ... inward facing head-mounted cameras”) and the face portion is a region of skin in an eye region within an eye-box of the HMD. ([para. 0122] “The locations ... of the ... head-mounted inward-facing cameras ... capture images of different respective first and second regions... skin on the user’s face”; [para. 0125] “the first region is located above the user’s eyes, and the second region is located below the user’s eyes”)
As per claim 4, Frank in view of Chao teaches claim 1.
Frank also teaches determining the current heartrate based on remote photoplethysmography (rPPG). ([Frank, para. 0171] “iPPG signals [rPPG – see para. 0061: “Other names known in the art for iPPG include ... remote photoplethysmography] extracted from the images ... analyzed to determine parameters such as heart rate”)
As per claim 5, Frank in view of Chao teaches claim 4.
Frank also teaches wherein determining the current heartrate comprises: ([Frank, para. 0061] “names known in the art for iPPG from the face include ... dynamic hemoglobin concentration/information extraction”; [para. 0171] “iPPG signals extracted from the images ... analyzed to determine parameters such as heart rate”)
extracting an average intensity over the face portion in the video signal to produce a raw signal; ([Frank, para. 0099] “a hemoglobin concentration pattern calculated from images [over the face portion in the video signal] refers to ... values [a raw signal] that are average intensities of one or more colors of the pixels over .... the images [extracting an average intensity over the face portion in the video signal]”)
filtering the raw signal to produce a filtered signal; ([Frank, para. 0064] “The analysis of PPG signals ... includes ... filtration of a PPG signal [filtering the raw signal to produce a filtered signal]”)
transforming the raw signal into a frequency domain to produce a transformed signal; and ([Frank, para. 0076] “The images from which the iPPG signal and/or the hemoglobin concentration patterns are extracted ... undergo various preprocessing to improve the signal, such as ... continuous wavelet transform (CWT) [a frequency domain transformation algorithm]”)
determining the current heartrate based on the transformed signal. ([Frank, para. 0399] “examples of physiological responses include ... heart rate”; [para. 0406] “detect the physiological response by comparing ... values derived therefrom ... the frequency domain [transformed signal]”)
As per claim 6, Frank in view of Chao teaches claim 5.
Frank also teaches the transformed signals are added as additional peaks corresponding to data values. ([Frank, para. 0064] “The analysis of PPG signals ... includes ... filtration of a PPG signal [transformed signal] ... extraction [add] of feature values from fiducial points [additional peaks] in the PPG signal [from the transformed signals]”; [para. 0067] “Fiducial points in the PPG signal ... include ... the systolic peak”; [para. 0171] “peak to peak pulse points ... analyzed to determine ... heart rate [data values]”)
Frank does not clearly teach wherein embedding the data comprises altering the face portion such that the transformed signals are added.
However, Chao teaches wherein embedding the data comprises altering the face portion such that the transformed signals are added. ([Chao, Fig. 1; para. 0018] “determination result 106 [the face portion] may include ... one or more of an image 106a captured by the camera, a face detection result 106b, a textual indication 106c of the determined heart rate, and a graphical indication 106d of the heart rate [altering the face portion such that the transformed signals/PPG signals are added]”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of Frank and Chao for the same reasons as disclosed above.
As per claim 7, Frank in view of Chao teaches claim 6.
Frank also teaches wherein the additional peaks have height values corresponding to discrete data values. ([Frank, para. 0099] “calculated from images ... values that are ... intensities [height values] of one or more colors of the pixels [discrete data values]”; [para. 0103] “the intensities being evaluated ... intensities corresponding to ... systolic peaks [additional peaks]”)
As per claim 8, Frank in view of Chao teaches claim 6.
Frank also teaches wherein height values of the additional peaks are dependent upon a height of a peak ([Frank, para. 0103] “a value representing the proportion of the area in which the intensities of pixels reach a threshold [a height of a peak] ... may be [maximum intensities corresponding to times of systolic peaks [height values of the additional peaks]”) corresponding to the heartrate. ([Frank, para. 0171] “peak to peak pulse points ... analyzed to determine ... heart rate [data values]”)
As per claim 9, Frank in view of Chao teaches claim 1.
Frank also teaches wherein a reading device captures images of the user wearing the wearable electronic device and determines the data based on the images. ([Frank, para. 0114] “first and second head-mounted inward-facing cameras 332 [a reading device] that take images 333 [captures images] of respective first and second regions on a user's face [of the user wearing the wearable electronic device]”; [para. 0171] “feature values ... derived from images ... to determine ... heart rate [the data]”)
As per claim 10, Frank in view of Chao teaches claim 9.
Frank also teaches wherein the reading device uses remote photoplethysmography (rPPG) to identify a heartrate in the captured images ([Frank, para. 0171] “iPPG signals [rPPG – see para. 0061: “Other names known in the art for iPPG include ... remote photoplethysmography] extracted from the images ... analyzed to determine parameters such as heart rate”) and uses the heartrate to determine the data based on the images. ([Para. 0320] “One sign of CHF is an increase heart rate [the data] ... the computer may utilize one or more feature values indicative of the user’s heart rate [uses the heart rate... derived from the images [based on the image]”)
As per claim 16, Frank in view of Chao teaches claim 1.
Frank also teaches wherein the electronic device is a head-mounted device (HMD) displaying the video signal to present a view of an eye region of the user. ([Frank, para. 0122] “Head-mounted inward-facing cameras [the electronic device] ... capture images of different respective first and second regions [displaying the video signal] ... skin on the user’s face”; [para. 0125] “the first region is located above the user’s eyes, and the second region is located below the user’s eyes [present a view of an eye region of the user]”)
As per claim 17, Frank teaches a wearable device comprising: ([Frank, para. 0027] “Embodiments ... implemented using various head-mounted objects that are worn on the user's head”)
a non-transitory computer-readable storage medium; ([Frank, para. 0089] “electronics that may be included in smartglasses and/or a smart-helmet include... a memory”)
one or more sensors; and ([Frank, para. 0089] “electronics that may be included in smartglasses and/or a smart-helmet include... sensors”)
one or more processors coupled to the non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium comprises program instructions that, when executed on the one or more processors, cause the one or more processors to perform operations comprising: ([Frank, para. 0530] “embodiments may be implemented ... in software ... involve a computer program that includes one or more instructions or code stored or transmitted on a computer-readable medium and executed by one or more processors”)
generating a video signal depicting a current appearance of a face portion, ([Frank, para. 0114] “first and second head-mounted inward-facing cameras 332 that take images 333 [generating a video signal] of respective first and second regions on a user's face [of a face portion]”; [para. 0135] “The current images [a current appearance of a face portion] are taken”) the video signal generated based on sensor data captured via the one or more sensors, ([para. 0089] “a camera housing that is physically coupled to a helmet, where the housing includes the imaging sensor”; [para. 0126] “each of these cameras is a multi-pixel video camera having a CMOS or a CCD sensor ... capture images [the video signal] at various rates [based on sensor data]”) wherein changes in an attribute of the face portion in the video signal over time correspond to a current heart rate of a user wearing the wearable electronic device. ([Para. 0061] “a non-contact camera that captures images of the skin, where a computer extracts the PPG signal [attribute of the face portion – see para. 0171: “one or more feature values indicative of the user’s heart rate ... derived from images ... by performing calculations on iPPG signals extracted from the images”] from the images [in the video signal] using an imaging photoplethysmography (iPPG) technique”; [para. 0183] “the computer 340 calculates, based on the current set of images, a current heart rate”; [para. 0320] “a time series signal [changes over time] is generated from video images of a subject’s exposed skin ... to obtain the PPG signal, which may be analyzed to determine ... heart rate”; the heart rate is of a user wearing the electronic device: see for example, Fig. 2b where the user wearing the electronic device is depicted)
Frank does not clearly teach embedding data into the video signal by altering the attribute of the face portion in the video signal over time such that the changes in the attribute of the face portion in the video signal over time correspond to both the current heart rate and the data; and presenting the video signal depicting the current appearance of the face portion on an outward-facing display of the wearable electronic device.
However, Chao teaches embedding data into the video signal ([Chao, para. 0015] “frontend 102 may access video data from a camera [the video signal] ... the user interface of frontend 102 may ... display determination result 106 to a user”; [Fig. 1; para. 0018] “determination result 106 may include ... one or more of an image 106a captured by the camera, [video signal] a face detection result 106b, a textual indication 106c of the determined heart rate, and a graphical indication 106d of the heart rate [data embedded into the video signal]”) by altering the attribute of the face portion in the video signal over time ([para. 0019] “To extract PPG signals ... one or more faces may be detected in the videos... detector's super-realtime performance [over time – see also para. 0013 describing the monitoring as continuous/over time] ... enable it to quickly locate an accurate facial region of interest (ROI) for PPG signal extraction [altering the attribute/PPG signal of the face portion in the video]”) such that the changes in the attribute of the face portion in the video signal over time correspond to both the current heart rate ([para. 0019] “record and/or process PPG signals, and may calculate heart rates based on the PPG signals”) and the data; and ([para. 0019] “determination result 106 ... include ... a textual indication 106c of the determined heart rate, and a graphical indication 106d of the heart rate”; as the changes in the attribute/PPG signal of the face over time are used to calculate the current heart rate, and the current heart rate is the data embedded into the video signals, the changes in the attribute necessarily correspond to “the data”)
presenting the video signal depicting the current appearance of the face portion ([Chao, para. 0015] “the user interface of frontend 102 may provide or display determination result 106 to a user [provide the current appearance of the face portion]”) on an outward-facing display of the wearable electronic device. ([Para. 0024] “user device 210 may perform one or more functions associated with frontend 102”; [para. 0025] “User device 210 includes ... a wearable device”; [para. 0040] the device includes a display that provides output [outward facing] information)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to have modified the elements disclosed by Frank with the teachings of Chao to include embedding data into the video signal by altering the attribute of the face portion in the video signal over time such that the changes in the attribute of the face portion in the video signal over time correspond to both the current heart rate and the data; and presenting the video signal depicting the current appearance of the face portion on an outward-facing display of the wearable electronic device. One of ordinary skill in the art would have been motivated to make this modification because such an application beneficially allows for monitoring of the graph and face in everyday scenarios like working in front of a computer. (Chao, para. 0011)
As per claim 18, the claim language is identical or substantially similar to that of claim 2. Therefore, it is rejected under the same rationale applied to claim 2.
As per claim 19, the claim language is identical or substantially similar to that of claim 3. Therefore, it is rejected under the same rationale applied to claim 3.
As per claim 20, Frank teaches a non-transitory computer-readable storage medium, storing program instructions executable on a device including one or more processors to perform operations comprising: ([Frank, para. 0530] “embodiments may be implemented ... in software ... involve a computer program that includes one or more instructions or code stored or transmitted on a computer-readable medium and executed by one or more processors”)
generating a video signal depicting a current appearance of a face portion, ([Frank, para. 0114] “first and second head-mounted inward-facing cameras 332 that take images 333 [generating a video signal] of respective first and second regions on a user's face [of a face portion]”; [para. 0135] “The current images [a current appearance of a face portion] are taken”) the video signal generated based on sensor data captured via one or more sensors, ([para. 0089] “a camera housing that is physically coupled to a helmet, where the housing includes the imaging sensor”; [para. 0126] “each of these cameras is a multi-pixel video camera having a CMOS or a CCD sensor ... capture images [the video signal] at various rates [based on sensor data]”) wherein changes in an attribute of the face portion in the video signal over time correspond to a current heart rate of a user wearing the wearable electronic device. ([Para. 0061] “a non-contact camera that captures images of the skin, where a computer extracts the PPG signal [attribute of the face portion – see para. 0171: “one or more feature values indicative of the user’s heart rate ... derived from images ... by performing calculations on iPPG signals extracted from the images”] from the images [in the video signal] using an imaging photoplethysmography (iPPG) technique”; [para. 0183] “the computer 340 calculates, based on the current set of images, a current heart rate”; [para. 0320] “a time series signal [changes over time] is generated from video images of a subject’s exposed skin ... to obtain the PPG signal, which may be analyzed to determine ... heart rate”; the heart rate is of a user wearing the electronic device: see for example, Fig. 2b where the user wearing the electronic device is depicted)
Frank does not clearly teach embedding data into the video signal by altering the attribute of the face portion in the video signal over time such that the changes in the attribute of the face portion in the video signal over time correspond to both the current heart rate and the data; and presenting the video signal depicting the current appearance of the face portion on an outward-facing display of the wearable electronic device.
However, Chao teaches embedding data into the video signal ([Chao, para. 0015] “frontend 102 may access video data from a camera [the video signal] ... the user interface of frontend 102 may ... display determination result 106 to a user”; [Fig. 1; para. 0018] “determination result 106 may include ... one or more of an image 106a captured by the camera, [video signal] a face detection result 106b, a textual indication 106c of the determined heart rate, and a graphical indication 106d of the heart rate [data embedded into the video signal]”) by altering the attribute of the face portion in the video signal over time ([para. 0019] “To extract PPG signals ... one or more faces may be detected in the videos... detector's super-realtime performance [over time – see also para. 0013 describing the monitoring as continuous/over time] ... enable it to quickly locate an accurate facial region of interest (ROI) for PPG signal extraction [altering the attribute/PPG signal of the face portion in the video]”) such that the changes in the attribute of the face portion in the video signal over time correspond to both the current heart rate ([para. 0019] “record and/or process PPG signals, and may calculate heart rates based on the PPG signals”) and the data; and ([para. 0019] “determination result 106 ... include ... a textual indication 106c of the determined heart rate, and a graphical indication 106d of the heart rate”; as the changes in the attribute/PPG signal of the face over time are used to calculate the current heart rate, and the current heart rate is the data embedded into the video signals, the changes in the attribute necessarily correspond to “the data”)
presenting the video signal depicting the current appearance of the face portion ([Chao, para. 0015] “the user interface of frontend 102 may provide or display determination result 106 to a user [provide the current appearance of the face portion]”) on an outward-facing display of the wearable electronic device. ([Para. 0024] “user device 210 may perform one or more functions associated with frontend 102”; [para. 0025] “User device 210 includes ... a wearable device”; [para. 0040] the device includes a display that provides output [outward facing] information)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to have modified the elements disclosed by Frank with the teachings of Chao to include embedding data into the video signal by altering the attribute of the face portion in the video signal over time such that the changes in the attribute of the face portion in the video signal over time correspond to both the current heart rate and the data; and presenting the video signal depicting the current appearance of the face portion on an outward-facing display of the wearable electronic device. One of ordinary skill in the art would have been motivated to make this modification because such an application beneficially allows for monitoring of the graph and face in everyday scenarios like working in front of a computer. (Chao, para. 0011)
Claims 11-14 are rejected under 35 U.S.C. 103 as being unpatentable over Frank in view of Chao and further in view of Rahmel et al. (US Pub. 2019/0347386) (hereinafter “Rahmel”)
As per claim 11, Frank in view of Chao teaches claim 1.
Frank also teaches wherein a reading device: ([Frank, para. 0114] “first and second head-mounted inward-facing cameras 332 [a reading device]”)
captures images of the user wearing the wearable electronic device; ([Frank, para. 0114] “first and second head-mounted inward-facing cameras 332 that take images 333 [captures images] of respective first and second regions on a user's face [of the user wearing the wearable electronic device”)
identifies a first patch of skin of the user wearing the wearable electronic device directly visible in the images; and ([Frank, para. 0122] “The locations ... of the ... head-mounted inward-facing cameras ... capture images [identifies] of different respective first [first patch of the user wearing the electronic devices] and second regions ... skin on the user’s face”)
identifies a second patch of skin of the user wearing the wearable electronic device in the video signal; ([Frank, para. 0122] “The locations ... of the ... head-mounted inward-facing cameras ... capture images [identifies in the video signal] of different respective first and second regions [second patch of the user wearing the electronic devices] ... skin on the user’s face”)
Frank does not clearly teach the video signal displayed on the outward-facing display of the wearable electronic device; compares heartrates identified from the first patch and the second patch; and decodes the data based on comparing the heartrates.
However, Chao teaches the video signal displayed ([Chao, para. 0015] “the user interface of frontend 102 may provide or display determination result 106 [the video signal – see Fig. 1] to a user”) on the outward-facing display of the wearable electronic device. ([Para. 0024] “user device 210 may perform one or more functions associated with frontend 102”; [para. 0025] “User device 210 includes ... a wearable device”; [para. 0040] the device includes a display that provides “output [outward facing] information”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of Frank and Chao for the same reasons as disclosed above.
Frank in view of Chao does not clearly teach compares heartrates identified from the first patch and the second patch; and decodes the data based on comparing the heartrates.
However, Rahmel teaches compares heartrates identified from the first patch and the second patch; and ([Rahmel, para. 0057-0059] “To authenticate himself ... the person places his index finger over ... the top PPG sensor 103 ... the top PPG sensor 103 is ... reading the person’s heart beat from the index finger ... At the same time, the bottom PPG sensor also monitors the person’s heart beat pattern from the person’s wrist ... the heart beat pattern read by the top PPG sensor 103 and heart beat pattern read by the bottom PPG sensor 211 should be identical”)
decodes the data based on comparing the heartrates. ([Rahmel, para. 0067] “once the person has been authenticated ... the server is able to decrypt the data”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to have modified the elements disclosed by Frank in view of Chao with the teachings of Rahmel to include compares heartrates identified from the first patch and the second patch; and decodes the data based on comparing the heartrates. One of ordinary skill in the art would have been motivated to make this modification because this allows improved identity security in that the set of physiologic data because the data is not pre-recorded. (Rahmel, para. 0022)
As per claim 12, Frank in view of Chao teaches claim 1.
Frank also teaches wherein a reading device: ([Frank, para. 0114] “first and second head-mounted inward-facing cameras 332 [a reading device]”)
captures images of the user wearing the wearable electronic device; ([Frank, para. 0114] “first and second head-mounted inward-facing cameras 332 that take images 333 [captures images] of respective first and second regions on a user's face [of the user wearing the wearable electronic device”)
identifies a first patch of skin of the user wearing the wearable electronic device directly visible in the images; and ([Frank, para. 0122] “The locations ... of the ... head-mounted inward-facing cameras ... capture images [identifies] of different respective first [first patch of the user wearing the electronic devices] and second regions ... skin on the user’s face”)
identifies a second patch of skin of the user wearing the wearable electronic device in the video signal. ([Frank, para. 0122] “The locations ... of the ... head-mounted inward-facing cameras ... capture images [identifies in the video signal] of different respective first and second regions [second patch of the user wearing the electronic devices] ... skin on the user’s face”)
Frank does not clearly teach the video signal displayed on the outward-facing display of the wearable electronic device; compares heartrates identified from the first patch and the second patch; and authenticates the user based on comparing the heartrates.
However, Chao teaches the video signal displayed ([Chao, para. 0015] “the user interface of frontend 102 may provide or display determination result 106 [the video signal – see Fig. 1] to a user”) on the outward-facing display of the wearable electronic device. ([Para. 0024] “user device 210 may perform one or more functions associated with frontend 102”; [para. 0025] “User device 210 includes ... a wearable device”; [para. 0040] the device includes a display that provides “output [outward facing] information”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of Frank and Chao for the same reasons as disclosed above.
Frank in view of Chao does not clearly teach compares heartrates identified from the first patch and the second patch; and authenticates the user based on comparing the heartrates.
However, Rahmel teaches compares heartrates identified from the first patch and the second patch; and ([Rahmel, para. 0057-0059] “To authenticate himself ... the person places his index finger over ... the top PPG sensor 103 ... the top PPG sensor 103 is ... reading the person’s heart beat from the index finger ... At the same time, the bottom PPG sensor also monitors the person’s heart beat pattern from the person’s wrist ... the heart beat pattern read by the top PPG sensor 103 and heart beat pattern read by the bottom PPG sensor 211 should be identical”)
authenticates the user based on comparing the heartrates. ([Rahmel, para. 0059] “if the heart beat patterns observed by each of the two PPG sensors in the embodiment 100 correlate, [based on comparing the heartbeats] it is deemed that the person who is wearing the embodiment 100 is also the person [authenticates the user] whose fingerprint has been scanned”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of Frank, Chao and Rahmel for the same reasons as disclosed above.
As per claim 13, Frank in view of Chao teaches claim 1.
Frank in view of Chao does not clearly teach further comprising using additional data from a third device to identify a heartrate of the user.
However, Rahmel teaches further comprising using additional data from a third device to identify a heartrate of the user. ([Rahmel, para. 0082-0083] “An ECG can identify ... the full heartbeat waveform ... The ECG sensor 801 [from a third device] ... observes the ECG of the person [using additional data] ... The heart rate”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to have modified the elements disclosed by Frank in view of Chao with the teachings of Rahmel to include further comprising using additional data from a third device to identify a heartrate of the user. One of ordinary skill in the art would have been motivated to make this modification because the PPG sensor is unable to collect fine details for biometric recognition like the ECG sensor but, on the other hand, is more rugged and robust, and is useable to monitor the heart beat pattern of the person over long period of time while he is engaging in strenuous activities and in contrast, a wearable ECG sensor is more delicate and may be obtaining too much information after the person has been verified or authenticated, which requires more intensive processing power, therefore, it is preferred to have both the PPG sensor and the ECG sensor in the same embedment. (Rahmel, para. 0083-0084)
As per claim 14, Frank in view of Chao teaches claim 3.
Frank in view of Chao does not clearly teach wherein the heartrate of the user identified from the additional data from the third device is used to confirm an identity of the user.
However, Rahmel teaches wherein the heartrate of the user identified from the additional data from the third device is used to confirm an identity of the user. ([Rahmel, para. 0082] “An ECG [the additional data from the third device] can identify a person using [confirm an identity of the user] ... the full heartbeat waveform [the heartrate of the user identified from the additional data from the third device]”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to combine the teachings of Frank, Chao and Rahmel for the same reasons as disclosed above.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Frank in view of Chao and further in view of Edwards et al. (US Pub. 2020/0213300) (hereinafter “Edwards”)
As per claim 15, Frank in view of Chao teaches claim 1.
Frank in view of Chao does not clearly teach wherein embedding the data into the video signal presented on the outward-facing display of the wearable electronic device enables another device to automatically connect or authenticate to share content with the wearable electronic device.
However, Edwards teaches wherein embedding the data into the video signal presented on the outward-facing display of the wearable electronic device ([Edwards, para. 0056] “display screen 404 [the outward-facing display of the wearable electronic device – see Fig. 4B and para. 0030] ... includes a graph 414 [the video signal] illustrating the changes to a client's resting heartrate over time [embedding the data into the video signal] ... When changes to the resting heartrate of the client exceed an allowable threshold for a pre-determined time period, regeneration of a heartrate profile for the client may be triggered”) enables another device to automatically connect or authenticate to share content with the wearable electronic device. ([para. 0056] “the heartrate profile to generate an expected heartrate range [enables, as the display screen/video signal triggers generation of the heartrate profile] against which a cardholder heartrate may be compared for authentication purposes”; [para. 0065] “if the cardholder heartrate corresponds to the expected heartrate ... the client is authenticated, and account access is enabled [to share content with the wearable electronic device – see para. 0029: “client devices ... to access service provider content and services”]”)
It would have been obvious before the effective filing date of the claimed invention for one of ordinary skill in the art to have modified the elements disclosed by Frank in view of Chao with the teachings of Edwards to include embedding the data into the video signal presented on the outward-facing display of the wearable electronic device enables another device to automatically connect or authenticate to share content with the wearable electronic device. One of ordinary skill in the art would have been motivated to make this modification because information generated by existing heartrate capture devices may advantageously be leveraged by authentication services to provide a low cost, non-invasive authentication solution. (Edwards, para.0007)
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Marks et al. (US Pub. 2025/0072773) discloses receiving a sequence of iPPG signals measured from different regions of a skin by a video image, transforming data from the video image by a FFT, and displaying the data on an external display.
Katto et al. (US Pub. 2024/0331846) discloses authentication using rPPG where a biometric identifier used for authentication is generated in accordance to a heart rate.
Rahman et al. (US Pub. 2012/0317024) discloses physiological signatures, such as a heart rate, may be used to create a key used to authenticate a user’s identity.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZHE LIU whose telephone number is (571) 272-3634. The examiner can normally be reached on Monday - Friday: 8:30 AM to 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Carl Colin can be reached on (571) 272-3862. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at (866) 217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call (800) 786-9199 (IN USA OR CANADA) or (571) 272-1000.
/ZHE LIU/Examiner, Art Unit 2493
Prosecution Insights