WO2014063160A1 - Detection of emotional states - Google Patents

Detection of emotional states Download PDF

Info

Publication number
WO2014063160A1
WO2014063160A1 PCT/US2013/065965 US2013065965W WO2014063160A1 WO 2014063160 A1 WO2014063160 A1 WO 2014063160A1 US 2013065965 W US2013065965 W US 2013065965W WO 2014063160 A1 WO2014063160 A1 WO 2014063160A1
Authority
WO
WIPO (PCT)
Prior art keywords
stress
user
stress state
data
features
Prior art date
Application number
PCT/US2013/065965
Other languages
French (fr)
Inventor
Nathan Ronald KOWAHL
Jonathan K. LEE
Marco Kenneth DELLA TORRE
Matthew Wayne ECKERLE
Jean Louise RINTOUL
Timothy MELANO
Original Assignee
Basis Science, Inc.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Basis Science, Inc. filed Critical Basis Science, Inc.
Priority to US14/436,975 priority Critical patent/US20150245777A1/en
Publication of WO2014063160A1 publication Critical patent/WO2014063160A1/en

Links

Classifications

    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/026Measuring blood flow
    • A61B5/0261Measuring blood flow using optical means, e.g. infrared light
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/0205Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/16Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165Evaluating the state of mind, e.g. depression, anxiety
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01FMEASURING VOLUME, VOLUME FLOW, MASS FLOW OR LIQUID LEVEL; METERING BY VOLUME
    • G01F1/00Measuring the volume flow or mass flow of fluid or fluent solid material wherein the fluid passes through a meter in a continuous flow
    • G01F1/704Measuring the volume flow or mass flow of fluid or fluent solid material wherein the fluid passes through a meter in a continuous flow using marked regions or existing inhomogeneities within the fluid stream, e.g. statistically occurring variations in a fluid parameter
    • G01F1/708Measuring the time taken to traverse a fixed distance
    • G01F1/7086Measuring the time taken to traverse a fixed distance using optical detecting arrangements
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/0205Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
    • A61B5/02055Simultaneously evaluating both cardiovascular condition and temperature
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024Detecting, measuring or recording pulse rate or heart rate
    • A61B5/02416Detecting, measuring or recording pulse rate or heart rate using photoplethysmograph signals, e.g. generated by infrared radiation
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024Detecting, measuring or recording pulse rate or heart rate
    • A61B5/02438Detecting, measuring or recording pulse rate or heart rate with portable devices, e.g. worn by the patient
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/103Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1118Determining activity level
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/68Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802Sensor mounted on worn items
    • A61B5/681Wristwatch-type devices

Definitions

  • the disclosure generally relates to the field of devices for measuring and characterizing stress.
  • FIG. 1 illustrates one embodiment of a wearable device for measuring stress.
  • FIG. 2 illustrates one embodiment of the wearable device for measuring stress according to a second view.
  • FIG. 3 illustrates a top view and a bottom of one embodiment of the wearable device for measuring stress.
  • FIG. 4 illustrates a more detailed view of the wearable device comprising an optical sensing system.
  • FIG. 5 illustrates a system architecture of the wearable device according to one embodiment.
  • FIG. 6 illustrates a method for identifying and characterizing stress according to one embodiment.
  • FIG. 7 illustrates exemplary signal acquisition schema for identifying and characterizing stress.
  • FIG. 8 illustrates raw signal acquired from optical sensors according one embodiment.
  • FIG. 9 illustrates processed signal acquired from optical sensors according one embodiment.
  • FIG. 10 illustrates processed signal acquired from optical sensors according one embodiment.
  • FIG. 11 illustrates a method of identifying and characterizing stress including motion mitigation according to one embodiment.
  • FIG. 12 illustrates a method of identifying and characterizing stress including motion mitigation according to one embodiment.
  • FIG. 13 illustrates a method of identifying and characterizing stress including multiple sensors according to one embodiment.
  • One embodiment of a disclosed device includes an optical sensing system to detect features of blood flow and identify and characterize a stress state of a user based on those blood flow features.
  • Light transmitted or reflected from tissue of the user is measured by an optical sensor.
  • a processor analyzes the received optical signal to identify features of the blood flow.
  • the stress state of the user is determined based on the identified features.
  • the stress state can be characterized according to a type of stress, a level of stress or both. Examples of the type of stress include relaxation, cognitive stress, emotional stress, physical, behavioral or general stress.
  • the level of stress is identified and is determined independently of whether the type of stress is identified. Stress events can also be identified. Identifying stress events comprises detecting a time or time range during which a particular a type of stress or level of stress have been identified.
  • An individual's stress state can be ascertained by analyzing the individual's heart beats.
  • the heart beats can be analyzed by analyzing the individual's blood flow.
  • the amount of blood in any vessel or artery varies with changes in heart beats as blood is pumped through the body.
  • the parameters of this variation also referred to features of the blood flow, are related to stress the user is experiencing. Variation in blood flow and hence the blood flow features can be determined optically.
  • the disclosed systems, devices apply these principals to determining stress of a user wearing a wearable device.
  • Wearable technology enables people to interact with technology in a convenient and continuous manner, since it can be present on the body in the context of all lifestyle activities.
  • An advantage of wearable technology extends to the device being able to measure the user and her surroundings continuously, as well as the ability to provide immediate information and feedback to the user at any time she has the device.
  • FIGS. 1 through 4 illustrate various views of a device for measuring stress according to the present disclosure.
  • FIG. 1 illustrates an example of a wearable device 100 configured to be in close proximity to or in contact with a user.
  • the device 100 may be worn on a user's appendage or portion thereof, e.g., an arm or a wrist.
  • a fastening system 102 fastens the device 100 to the user's appendage.
  • the fastening system 102 may be removable, exchangeable, or customizable.
  • embodiments are described herein with respect to a wrist-worn device, other form factors or designed wear locations of the wearable device 100 may alternatively be used.
  • the wearable device 100 is a physiological monitoring device for monitoring activities of its wearer and calculating various physiological and kinematic parameters, such as activity levels, caloric expenditure, step counts, heart-rate, and sleep patterns.
  • the wearable device 100 includes a display 104 and several user interaction points.
  • the display 104 and user interaction points may be separate components of the device 100, or may be a single component.
  • the display 104 may be a touch-sensitive display configured to receive user touch inputs and display information to the user.
  • the wearable device may also have a display element such as 104 without interaction points, or interaction points without a display element such as 104.
  • the device 100 comprises one or more processors 101 and an optical sensing system 203.
  • Example processors 101 include the TIMSP430 from Texas Instruments and ARM Cortex-M class microcontrollers.
  • the processor 101 is configured to use signals received from the optical sensing system 203 to determine an emotional state of the user at or around the time the signals were measured.
  • the processor is further configured to determine a type, level and other parameters of the determined emotional state.
  • FIG. 2 illustrates a second view of the device 100 comprising the fastening system 102, processor 101 and optical sensing system 203.
  • FIG. 3 illustrates a top view and a bottom view of the device 100.
  • the processor 101 is inside the device and the optical sensing system 203 is visible on the bottom view of device 100.
  • the side illustrated in the bottom view is the side in touch with the skin when device 100 is worn.
  • FIG. 4 it illustrates the optical sensing system 203 according to one example embodiment.
  • the system 203 comprises an optical sensor 405 and two optical emitters 407. Embodiments with one optical emitter 407, more than two optical emitters 407 or more than one optical sensor 405 are also possible. Signals from the optical sensor 405 are transmitted to the processor 101.
  • Optical emitters 407 include light emitting diodes (LEDs) and lasers. In some embodiments light emitted from the optical emitters 407 is in the visible yellow and green ranges (500-600 nm). In other
  • Optical emitters 407 may emit light in other wavelengths in addition to those used for identifying blood flow features. For example, emitted light may be full spectrum white light.
  • optical emitters 407 emit light at the same time, but in another embodiment the optical emitters 407 emit light in an alternating fashion. In another embodiment the optical emitters 407 may be set to emit light independently at some times and simultaneously at others.
  • Optical sensor 405 detects light in the wavelengths of light emitted by the optical emitter 407.
  • An example optical sensor 405 is Light To Voltage (LTV) sensor such as a Taos TSL13T or similar.
  • LTV Light To Voltage
  • FIG. 5 illustrates a system architecture for the device 100.
  • the device 100 comprises the processor 101, display 104, optical sensor 405, optical emitters 407, memory 509, storage 511, user input 513 and transmission interface 515.
  • the memory 509 stores signals measured from the optical sensor 405 and parameters determined by the processor 101 from the measured signals, such as, for example, measurements of stress.
  • FIG. 6 illustrated is an example method for acquiring signals from the optical sensor 405 and determining parameters about the emotional state of the user at or around the time the signals were acquired. Specifically the method identifies and characterizes the user's stress state.
  • optical sensing system 203 is not continuously active. In some embodiments, the optical sensing system 203 generates data at intervals. The intervals may be regular or irregular.
  • the processor 101 may activate the optical sensing system 203 in response to a stimulus such as the user waking up or the user can request a stress state determination at a particular time or on demand. In some embodiments, the optical sensing system 203 activates at regular intervals to generate data and additionally it activates in response to a stimulus.
  • FIG. 7 illustrates example light emission and sensor sampling schemes.
  • both optical emitters 407 emit light onto the skin at the same time.
  • Line 701 illustrates activity of the optical sensor 405 and line 702 illustrates activity of the optical emitters 407. In each case the line illustrates when the respective component is on and off.
  • Line 703 illustrates the sampling rate.
  • the sampling frequency is between 2Hz-4096Hz, 20 Hz- 1024 Hz, 30 Hz- 1000 Hz, 50 Hz-512 Hz, 64 Hz-512 Hz, 100 Hz-256 Hz or 128 Hz-200 Hz.
  • the sampling frequency is 20, 30, 32, 50, 64, 100, 128, 200, 256, 500, 512, 1000 or 1024 Hz.
  • the optical emitters 407 emit light onto the skin at different times in an alternating fashion.
  • Line 704 illustrates activity of the optical sensor 405 and lines 705 and 706 each illustrate activity of one of the optical emitters 407. In each case the line illustrates when the respective component is on and off.
  • Lines 707 illustrate the sampling rate.
  • the sampling frequency for each of the optical emitters 407 is between 2Hz-4096Hz, 20 Hz-1024 Hz, 30 Hz-1000 Hz, 50 Hz-512 Hz, 64 Hz-512 Hz, 100 Hz-256 Hz or 128 Hz- 200 Hz.
  • the sampling frequency is 20, 30, 32, 50, 64, 100, 128, 200, 256, 500, 512, 1000 or 1024 Hz. In other embodiments, a combination of the two sampling schemes is utilized.
  • FIG. 8 illustrates an example of an acquired optical signal.
  • the signal has several main constituents - a large, low frequency signal, a smaller, higher frequency signal and still smaller signal representing noise.
  • the first two of these signals are analyzed because the levels of these two signals vary in response to the type or level of stress a user has experienced.
  • pre-processing 605 comprises a filtering leaving the low frequency, large amplitude signal.
  • Large, low frequency signals in the range 0.05-0.12Hz are examined for frequency, magnitude, consistency of variance, relative power and other parameters.
  • FIG. 9 illustrates determining some of these parameters. The intervals between subsequent peaks [ i ls i 3 ], and subsequent valleys [ i 2 ], can be used to derive
  • instantaneous and average frequency measures The height from peak to valley [ hi,h 3 ], and valley to peak [ h 2 ,h 4 ], may also be used as instantaneous or averaged magnitude measures.
  • pre-processing 605 comprises filtering leaving the higher frequency, smaller amplitude signal.
  • Smaller, higher frequency signals in the range 0.5-2Hz are examined for frequency, magnitude, consistency and other parameters.
  • FIG. 10 illustrates determining some of these parameters.
  • the intervals between subsequent peaks [ ii,i 3 ..i9 ], and subsequent valleys [ i 2 ,i 4 ..ii 0 ] can be used to derive instantaneous and average frequency measures.
  • the height from peak to valley [ hi,h 3 ..hn ], and valley to peak [ h 2 ,h 4 ..hi 0 ] may also be used as instantaneous or averaged magnitude measures. There are also techniques for measuring these features on an unfiltered signal.
  • These features relate to the heartbeat.
  • the size and variance of these peaks may relate to a wide range of cardiovascular parameters that are modulated by emotional changes in the body. Since these peaks relate to blood flow features such as the peak absorption level observed by the sensor, changes in these signals can relate to cardiovascular changes due to emotional states.
  • preprocessing can include assessing the quality and/or quantity of the collected data. In some embodiments this comprises determining an amount of time or a number of heart beats between interruptions to the data. Data collection can be interrupted by interference in the optical signals due to motion, for example.
  • Table 1 lists exemplary blood flow features that can be used to determine a user's stress state. In some embodiments, these blood flow features are determined from the optical signals. Table 1 includes four categories of blood flow features that can be determined from the optical signal. The features in two of these categories, 2 and 4, are features that are not available via electrocardiogram (ECG), a more conventional method used to assess stress states in individuals. These signals relate to peripheral vascular activity that can be indicative of the impacts of emotional states on the body. The features in category 1 can be determined via ECG and thus in other embodiments, if blood flow features from category 1 are used to determine stress states, the input data can be from ECG in addition to or in place of optical data.
  • ECG electrocardiogram
  • NN50 count Number of RR intervals exceeding 50 ms within the measurement time window. pNN50 NN50 count divided by the total number of all RR
  • NN20 count Number of RR intervals exceeding 20 ms within the measurement time window. pNN20 NN20 count divided by the total number of all RR
  • Peak height May alternatively use valley to peak height, or both.
  • Peaks in the 0.3Hz to 3.5Hz range only are considered, specifically those corresponding to frequencies near the heart rate.
  • VLF Very Low Frequency
  • LF Normalized LF power in normalized units LF/(Total Power- VLF)* 100.
  • HF Normalized HF power in normalized units HF/(Total Power- VLF)* 100.
  • intervals into discrete ranges [ 50 - 1000ms each ] and divide the number of entries in the modal range [ i.e. the range with the most number of entries ] by the sum of the number of entries in all the other ranges.
  • the identified blood flow features are used to determine 607 information about the user's stress status.
  • a pre -trained algorithm is used to convert the observed feature levels to an observed stress type, level or other descriptor.
  • the algorithm is trained in advance using data collected from subjects experiencing the types, levels and other categories of stress to be detected.
  • thresholds for various blood flow features and combinations of each feature's level are used. For example, if a certain proportion of features are beyond a threshold set to indicate stress, the data analyzed could then be considered to represent an elevated stress state.
  • Each of the above-identified methods for determining 607 the user's stress status can be normalized for individual users.
  • resting heart rate is determined for a user after the device is purchased and activated. That resting heart rate is used to normalize the data for the user. Normalizing the process for the user is described in further detail in Example 2.
  • processor 101 determines a quality or quantity of the collected optical data and determines the user's stress state only when the quality and/or quantity of data exceeds a threshold.
  • the thresholds for sufficient quantity or quality of data can be different based on the context of the user as the data is collected. If a user is moving vigorously, fewer of the user's heartbeats may be collected due to interference in the optical signal from motion.
  • the processor 101 uses data from the majority of the user's heartbeats in a given period of time (for example 3 or so minutes). However if the processor 101 determines that the user is moving, the processor 101 applies a lower threshold as to what is sufficient data to determine the user's stress state. While the assessment of the stress state may not be as accurate as when the higher threshold is applied, it is more useful to the user to have an assessment of stress state as opposed to no assessment.
  • the determined stress state is stored and is displayed to a user via the display 104 in response to a query received via the user interaction points on the face of the device 100. Additionally or alternatively, the stress status is displayed on the display 104 automatically. The status may be displayed automatically if the level of stress determined by the processor 101 exceeds a threshold. Providing the information to the user automatically is useful to alert the user that she is experiencing a high amount of stress. The processor 101 may also provide for display information to assist the user in reducing stress.
  • alternative biological parameters are used in combination with the blood flow features in the determination of an individual's stress state.
  • Table 2 identifies exemplary parameters that can also be used.
  • Temperature difference Difference between skin and ambient temperature be it the minimum, maximum, average, range, or standard deviation over time.
  • the presence of changes may also be used to signify a temperature difference event, which can then be used as a variable.
  • Blood Pressure Blood pressure minimum, maximum, average, range or standard deviation over time May be estimated from the optical data via low-frequency signal oscillations or pulse speed measurements. The presence of changes may also be used to signify a blood pressure event, which can then be used as a variable.
  • Electrodermal Activity Skin conductance changes over a time window
  • Patterns such as the ration of the time to rise versus the time to fall.
  • data is collected from users experiencing various stress types, levels of stress and stress events, a data set for each stress type to be detected can be prepared. These data sets are then used to train a classifier such as a Support Vector Machine ( SVM ), Random Forest or other machine learning techniques.
  • SVM Support Vector Machine
  • the classifier can be trained with all or some of the features of Tables 1 and 2. When data is presented to the system from a user, it can be decomposed into the features used to train the classifier which in turn allow for classification of the stress state of the user.
  • determination of a user's stress state may occur at a remote processor instead of on device 100.
  • the remote processor implements the functionality of processor 101.
  • the process described in FIG. 6 for determining a user's stress state can be apportioned between processor 101 and the remote processor in various ways.
  • device 100 transmits raw signal acquired from the optical sensor 407 to the remote processor and all processing is accomplished on the remote processor.
  • identified features of blood flow are transmitted to the remote processor. Transfer of the process between the device 100 and remote processor can occur at any step in between as well.
  • the received stress data and determined stress states can be stored remotely regardless of where the stress states were determined.
  • the information can be stored on a remote server so that the user can access the information via multiple devices such as a laptop computer, tablet computer, smartphone and the like.
  • Guidance for the user on managing her stress states can then be provided to the user on these other devices. This is useful as the display 104 on device 100 has limited space for displaying information.
  • the remote server may store various other information about the user such as a calendar application.
  • the remote server may match the time stamps from the collected data and determined stress state to calendar entries for the user and display the user's stress states during a given time period next to the calendar from that time period.
  • the remote processor may also instruct processor 101 to activate the optical sensing system 203 during times when the calendar application indicates there is an appointment. This is more beneficial to a user than having the stress state only determined at predetermined intervals such as every half hour.
  • Phase 1 Subjects were allowed to relax in a calm setting, with minimal ambient noise and in comfortable seating. They were instructed to enjoy a calm, relaxing time while staying awake.
  • Phase 2 Subjects engaged in a series of cognitive exercises under time pressure to induce a state of cognitive stress.
  • Phase 3 - Phase 1 was repeated to induce a state of calm.
  • the timing of each phase was recorded so that the biometric data from each phase could be identified and labeled with the specific emotional state of the subject.
  • the data recorded in this study includes data from a wearable device such as that described above, as well as other devices.
  • Electro-dermal activity (EDA) from a second site if a wrist- worn device was used, a finger may be used to collect EDA.
  • the first three minutes of the calm phase could be a time window, as could the three minutes between the first and fourth minute.
  • the second time window overlaps the first by one minute. If an overlapping system is used, the overlap can be as much as one second less than the window size, or as small as one second.
  • classifiers may be used, such as, but not limited to, Linear Discriminant, Support Vector Machine, Linear Regression or Neural Network.
  • a combination of classifiers may be optimal, since different class combinations may be more optimally separated by different classifier architectures. Different classifier architectures may be best at separating classes with different feature sets. For example, even when classifying the same classes, the features that are optimal for a Random Forest classifier may not be the same as features that perform best if a Linear Discriminant is used.
  • the time window that is used to train the classifier may be different to that used to evaluate a stress level using that classifier. For example, a classifier was trained using 3 minute time windows, but was used to evaluate recordings of just 1 minute duration.
  • the likelihood may be represented as a level, for example the degree to which the user is in one state versus another, or as a means to detect events over time. Events may be detected by thresholds that identify a change in likelihood over a period of time. For example, if a stress likelihood score from a range of 0 to 1 were to increase by more than 0.25 over 5 minutes, this could be classified as a stress event.
  • feature data can be sourced from multiple sensors
  • a different sensor may be used to train the algorithm than to operate it as a detector.
  • one feature used was Heart Rate Mean & Std, however the algorithm was trained using heart beat data from the ECG and then used as a detector via heart beat information sourced from a wearable optical sensor.
  • Normalization method parameters may include biasing terms (addition and subtraction), scaling terms (multiplication and division), or other non-linear processing parameters such as raising variables to a given power, and remapping the feature space via logarithmic, exponential or logistical transforms. Normalization parameters may be derived from scientific literature, or dynamically from the data itself using statistical or unsupervised learning techniques.
  • one feature that may be important in assessing the presence of an emotional arousal state is an increase in heart rate magnitude. Since different subjects may naturally have different heart rate magnitude levels when calm or aroused, it may be necessary to normalize a subject's observed heart rate magnitude.
  • the user's data is recorded during a 24 hour period and this data is used to generate a biasing and scaling term.
  • a median and standard deviation of heart rate magnitude during relatively inactive periods could be used to normalize for an individual. By subtracting the median from the subject's observed heart rate magnitude values and dividing by the standard deviation, the observed measures of heart rate magnitude may be normalized to similar levels for all subjects.
  • heart rate is scaled against an estimate of the user's maximum heart rate.
  • One estimate for maximum heart rate is 220-(user's age).
  • Another feature that may be important in assessing the presence of an emotional arousal state is a decrease in heart rate variance. Since different subjects may naturally have different heart rate variance levels when calm or aroused, it is possible to normalize by subtracting a subject's baseline heart rate variance level. In one
  • the user's data is recorded during sleep and this data is used to generate a baseline.
  • a median of inter-beat intervals during sleep could be used by subtracting this value from observed measures of inter-beat intervals during the day.
  • the observed heart rate magnitude will be biased by the average of the centroids given by the unsupervised learning method, and scaled by the magnitude of the centroid distances from each other.
  • a users' data may again be normalized before being processed by a classifier to correct for their personal baseline, maximum, or range.
  • the survey results after each phase could be used to weight the time windows during training.
  • the responses to a question asking the level of relaxation could be used as follows:
  • Cortisol is a hormone released in response to stress, and since hormone release can take much longer than the duration of a stimulus, measurement over time, after the study is complete, can give more insight into how stressful a study was for a subject.
  • a classifier Once a classifier has been trained, its operation can be updated in an ongoing basis to better match a single user's biosignals over time. For example, as baseline measures of cardiac function such as resting heart rate and heart rate variance change, the normalization process that is used to generate more consistent features can also evolve. In this way, the algorithm continues to adapt to a user over time and, thereby, maintains the accuracy of its detection of emotional states over time.
  • the system measures the resting heart rate by estimating it during and around times of sleep. This measurement is then used to normalize a feature based on heart rate, by subtracting the most recent resting heart rate measurement. In this way, while a user's heart rate may change, the system is constantly updating their resting heart rate value and normalizing their heart rate based feature with that most recent measurement of resting heart rate. Not only does this normalize across different subjects, but also across time for a single subject.
  • the optical signals related to blood flow information include noise introduced by motion of the wearer.
  • motion sensing may be used.
  • Example motion sensors include an accelerometer, gyroscope, pressure sensor, compass and magnetometer.
  • FIG. 11 shows an example of how such motion mitigation may be used.
  • Each sample from an accelerometer 1101 are evaluated 1102 against filtered version of historical values 1103. These differences are summed 1104 over a whole second and evaluated 1105 against a motion threshold 1106 to determine the level of motion contamination for the last second. This level can be used for the determination of whether the data should be used in further processing or not.
  • a device can evaluate the data being collected via its sensors to determine if sufficient data has been collected (both in quantity and quality) for the evaluation of the user's stress level.
  • FIGS. 12 and 13 illustrate examples of dynamic sensor and feature selection.
  • FIG. 12 illustrates activity parameters, such as the level of motion as detected by an accelerometer, evaluated 1201 against parameters indicating physical effort. Recent data is characterized as being related to physical effort 1202 or not 1203. The subsequent analysis of stress is given a different context for each of non- physical stress 1204 and physical stress 1205. An example of this would be the detection that the user has been on a recent run.
  • the processor 101 would operate in "non-physical stress" mode and discount data that might otherwise be considered as an indication of stress because after these data are consistent with ordinary response to physical exertion. Elevated heart rate, for example, would be associated with the physical effort of the run rather than a stress event.
  • FIG. 13 illustrates a second example of how multiple sensors could be used to modify the detection of stress parameters. Skin surface sensors can also be used.
  • Examples include galvanic skin response sensors, perspiration sensors and sweat constituent (Cortisol, alcohol, adrenalin, glucose, urea, ammonia, lactate) analysis.
  • a galvanic skin response sensor 1301 is used to detect a stress event.
  • optical sensor data is analyzed to characterize 1302 the stress at that time.
  • Optical sensor data is saved 1303 in a format allowing analysis of parameters upon detection of a galvanic skin response event.
  • the combined output of this algorithm would be both timing information 1304, as provided by the galvanic skin response event, and characterization as provided by the optical sensor data analysis.
  • the galvanic skin response sensor may also be used as an input to the stress characterization algorithm.
  • Additional sensors that can be used to provide additional functionality include environmental sensors (e.g., sensors for ultraviolet light, visible light, moisture/humidity, air quality including sensors to detect pollen, dust and other allergens).
  • environmental sensors e.g., sensors for ultraviolet light, visible light, moisture/humidity, air quality including sensors to detect pollen, dust and other allergens.
  • the disclosed embodiments beneficially allow for monitoring of a user's stress state over extended periods of time because the device collecting the data is worn by the user and is unobtrusive allowing for the device to be worn continually through most daily activities. Using the additional data collected, the system can determine a context for the individuals and thus provide a more personalized assessment of the user's stress. The more personalized assessment includes providing stress information in relation to the user's own baseline as well as using different methods to determine stress based on the user's activity level. [0084]
  • any reference to "one embodiment” or “an embodiment” means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment.
  • Coupled and “connected” along with their derivatives. It should be understood that these terms are not intended as synonyms for each other. For example, some embodiments may be described using the term “connected” to indicate that two or more elements are in direct physical or electrical contact with each other. In another example, some embodiments may be described using the term “coupled” to indicate that two or more elements are in direct physical or electrical contact. The term “coupled,” however, may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other. The embodiments are not limited in this context.
  • the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion.
  • a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
  • "or” refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).

Abstract

A system and a method are disclosed for identifying and characterizing a stress state of a user based on features of blood flow identified from optical signals. One embodiment of a disclosed system (and method) includes an optical sensing system to detect features of blood flow and identify and characterize a stress state of a user based on those blood flow features. Light transmitted or reflected from tissue of the user is measured by an optical sensor. A processor analyzes the received optical signal to identify features of the blood flow. The stress state of the user is determined based on the identified features. The stress state is characterized according to a type of stress, a level of stress or both. Additionally stress events are identified.

Description

DETECTION OF EMOTIONAL STATES
CROSS REFERENCE TO RELATED APPLICATIONS
[0001] This application claims the benefit of U.S. Provisional Application No.
61/716,405, filed October 19, 2012, which is incorporated by reference in its entirety.
BACKGROUND
1. FIELD OF ART
[0002] The disclosure generally relates to the field of devices for measuring and characterizing stress.
2. DESCRIPTION OF THE RELATED ART
[0003] Stress is well known to be a major contributor to overall health. CConvenient, continuous stress-monitoring devices that can be worn continuously and thus provide continual monitoring are currently lacking. Current devices require a user to be tethered to a computer or monitoring device via wires needed for the monitoring apparatus to communicate with the computer or monitoring device. Measurement of stress levels is also available through the collection and analysis of bodily fluids such as sweat, saliva or blood.
BRIEF DESCRIPTION OF DRAWINGS
[0004] The disclosed embodiments have advantages and features which will be more readily apparent from the detailed description, the appended claims, and the
accompanying figures (or drawings). A brief introduction of the figures is below.
[0005] Figure (FIG.) 1 illustrates one embodiment of a wearable device for measuring stress.
[0006] FIG. 2 illustrates one embodiment of the wearable device for measuring stress according to a second view.
[0007] FIG. 3 illustrates a top view and a bottom of one embodiment of the wearable device for measuring stress.
[0008] FIG. 4 illustrates a more detailed view of the wearable device comprising an optical sensing system.
[0009] FIG. 5 illustrates a system architecture of the wearable device according to one embodiment.
[0010] FIG. 6 illustrates a method for identifying and characterizing stress according to one embodiment.
[0011] FIG. 7 illustrates exemplary signal acquisition schema for identifying and characterizing stress.
[0012] FIG. 8 illustrates raw signal acquired from optical sensors according one embodiment.
[0013] FIG. 9 illustrates processed signal acquired from optical sensors according one embodiment.
[0014] FIG. 10 illustrates processed signal acquired from optical sensors according one embodiment.
[0015] FIG. 11 illustrates a method of identifying and characterizing stress including motion mitigation according to one embodiment.
[0016] FIG. 12 illustrates a method of identifying and characterizing stress including motion mitigation according to one embodiment.
[0017] FIG. 13 illustrates a method of identifying and characterizing stress including multiple sensors according to one embodiment.
DETAILED DESCRIPTION
[0018] The Figures (FIGS.) and the following description relate to preferred embodiments by way of illustration only. It should be noted that from the following discussion, alternative embodiments of the structures and methods disclosed herein will be readily recognized as viable alternatives that may be employed without departing from the principles of what is claimed.
[0019] Reference will now be made in detail to several embodiments, examples of which are illustrated in the accompanying figures. The figures depict embodiments of the disclosed system (or method) for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein may be employed without departing from the principles described herein.
CONFIGURATION OVERVIEW
[0020] One embodiment of a disclosed device (and method) includes an optical sensing system to detect features of blood flow and identify and characterize a stress state of a user based on those blood flow features. Light transmitted or reflected from tissue of the user is measured by an optical sensor. A processor analyzes the received optical signal to identify features of the blood flow. The stress state of the user is determined based on the identified features. The stress state can be characterized according to a type of stress, a level of stress or both. Examples of the type of stress include relaxation, cognitive stress, emotional stress, physical, behavioral or general stress. The level of stress is identified and is determined independently of whether the type of stress is identified. Stress events can also be identified. Identifying stress events comprises detecting a time or time range during which a particular a type of stress or level of stress have been identified.
[0021] An individual's stress state can be ascertained by analyzing the individual's heart beats. The heart beats can be analyzed by analyzing the individual's blood flow. The amount of blood in any vessel or artery varies with changes in heart beats as blood is pumped through the body. The parameters of this variation, also referred to features of the blood flow, are related to stress the user is experiencing. Variation in blood flow and hence the blood flow features can be determined optically.
[0022] Light having a wavelength that is absorbed by blood is emitted onto the skin and into the underlying tissue. Blood traveling through the skin and tissue will absorb a portion of the emitted light. Some of the remaining light is reflected back and some of the remaining light continues through the tissue. If the body part onto which the light is emitted is small enough (like for example an ear lobe or fingertip), some of the light will be transmitted all the way through the body. Both the amounts of transmitted and reflected light are a function of how much light was emitted and how much light was absorbed by the blood. Thus measuring the transmitted or reflected light allows a determination the amount of blood present in the tissue at the time the light passed through. Sampling the transmitted and/or reflected light over a period of time provides information about the features of the blood flow.
[0023] The disclosed systems, devices (and methods) apply these principals to determining stress of a user wearing a wearable device. A processor on the wearable device or located remotely determines the stress state. Wearable technology enables people to interact with technology in a convenient and continuous manner, since it can be present on the body in the context of all lifestyle activities. An advantage of wearable technology extends to the device being able to measure the user and her surroundings continuously, as well as the ability to provide immediate information and feedback to the user at any time she has the device.
DEVICE
[0024] FIGS. 1 through 4 illustrate various views of a device for measuring stress according to the present disclosure. FIG. 1 illustrates an example of a wearable device 100 configured to be in close proximity to or in contact with a user. For example, the device 100 may be worn on a user's appendage or portion thereof, e.g., an arm or a wrist. A fastening system 102 fastens the device 100 to the user's appendage. The fastening system 102 may be removable, exchangeable, or customizable. Furthermore, although embodiments are described herein with respect to a wrist-worn device, other form factors or designed wear locations of the wearable device 100 may alternatively be used. For example, embodiments described herein may be implemented in arm-worn devices, chest- worn devices, head-worn devices, clip-on devices, and so forth. In one embodiment, the wearable device 100 is a physiological monitoring device for monitoring activities of its wearer and calculating various physiological and kinematic parameters, such as activity levels, caloric expenditure, step counts, heart-rate, and sleep patterns.
[0025] The wearable device 100 includes a display 104 and several user interaction points. The display 104 and user interaction points may be separate components of the device 100, or may be a single component. For example, the display 104 may be a touch- sensitive display configured to receive user touch inputs and display information to the user. The wearable device may also have a display element such as 104 without interaction points, or interaction points without a display element such as 104.
[0026] Generally, the device 100 comprises one or more processors 101 and an optical sensing system 203. Example processors 101 include the TIMSP430 from Texas Instruments and ARM Cortex-M class microcontrollers. The processor 101 is configured to use signals received from the optical sensing system 203 to determine an emotional state of the user at or around the time the signals were measured. The processor is further configured to determine a type, level and other parameters of the determined emotional state.
[0027] FIG. 2 illustrates a second view of the device 100 comprising the fastening system 102, processor 101 and optical sensing system 203. FIG. 3 illustrates a top view and a bottom view of the device 100. The processor 101 is inside the device and the optical sensing system 203 is visible on the bottom view of device 100. The side illustrated in the bottom view is the side in touch with the skin when device 100 is worn.
[0028] Turning now to FIG. 4, it illustrates the optical sensing system 203 according to one example embodiment. The system 203 comprises an optical sensor 405 and two optical emitters 407. Embodiments with one optical emitter 407, more than two optical emitters 407 or more than one optical sensor 405 are also possible. Signals from the optical sensor 405 are transmitted to the processor 101. Optical emitters 407 include light emitting diodes (LEDs) and lasers. In some embodiments light emitted from the optical emitters 407 is in the visible yellow and green ranges (500-600 nm). In other
embodimentslight in the visible spectrum, such as blue and red, or the infrared spectrum may be used instead of or in addition to green light. Optical emitters 407 may emit light in other wavelengths in addition to those used for identifying blood flow features. For example, emitted light may be full spectrum white light.
[0029] In one embodiment, optical emitters 407 emit light at the same time, but in another embodiment the optical emitters 407 emit light in an alternating fashion. In another embodiment the optical emitters 407 may be set to emit light independently at some times and simultaneously at others. Optical sensor 405 detects light in the wavelengths of light emitted by the optical emitter 407. An example optical sensor 405 is Light To Voltage (LTV) sensor such as a Taos TSL13T or similar.
[0030] FIG. 5 illustrates a system architecture for the device 100. The device 100 comprises the processor 101, display 104, optical sensor 405, optical emitters 407, memory 509, storage 511, user input 513 and transmission interface 515. The memory 509 stores signals measured from the optical sensor 405 and parameters determined by the processor 101 from the measured signals, such as, for example, measurements of stress.
EXAMPLE PROCESSING CONFIGURATION
[0031] Referring now to Figures 6 through 13, illustrated is an example method for acquiring signals from the optical sensor 405 and determining parameters about the emotional state of the user at or around the time the signals were acquired. Specifically the method identifies and characterizes the user's stress state.
[0032] Referring first to FIG. 6, light is emitted 601 from the optical emitters 407. An optical signal is acquired 603 via the optical sensor 405. The emission of light and subsequent collection of signals can take place continuously while the device is worn by the user. Whether the device is being worn can be assessed through any means known in the art, including, but not limited to, use of a proximity sensor. Alternatively, the optical sensing system 203 is not continuously active. In some embodiments, the optical sensing system 203 generates data at intervals. The intervals may be regular or irregular. The processor 101 may activate the optical sensing system 203 in response to a stimulus such as the user waking up or the user can request a stress state determination at a particular time or on demand. In some embodiments, the optical sensing system 203 activates at regular intervals to generate data and additionally it activates in response to a stimulus.
[0033] FIG. 7 illustrates example light emission and sensor sampling schemes. In a first embodiment, shown in the top graph, both optical emitters 407 emit light onto the skin at the same time. Line 701 illustrates activity of the optical sensor 405 and line 702 illustrates activity of the optical emitters 407. In each case the line illustrates when the respective component is on and off. Line 703 illustrates the sampling rate. In some embodiments, the sampling frequency is between 2Hz-4096Hz, 20 Hz- 1024 Hz, 30 Hz- 1000 Hz, 50 Hz-512 Hz, 64 Hz-512 Hz, 100 Hz-256 Hz or 128 Hz-200 Hz. In some embodiments the sampling frequency is 20, 30, 32, 50, 64, 100, 128, 200, 256, 500, 512, 1000 or 1024 Hz.
[0034] In a second embodiment illustrated the bottom graph of FIG. 7, the optical emitters 407 emit light onto the skin at different times in an alternating fashion. Line 704 illustrates activity of the optical sensor 405 and lines 705 and 706 each illustrate activity of one of the optical emitters 407. In each case the line illustrates when the respective component is on and off. Lines 707 illustrate the sampling rate. In some embodiments the sampling frequency for each of the optical emitters 407 is between 2Hz-4096Hz, 20 Hz-1024 Hz, 30 Hz-1000 Hz, 50 Hz-512 Hz, 64 Hz-512 Hz, 100 Hz-256 Hz or 128 Hz- 200 Hz. In some embodiments the sampling frequency is 20, 30, 32, 50, 64, 100, 128, 200, 256, 500, 512, 1000 or 1024 Hz. In other embodiments, a combination of the two sampling schemes is utilized.
[0035] FIG. 8 illustrates an example of an acquired optical signal. The signal has several main constituents - a large, low frequency signal, a smaller, higher frequency signal and still smaller signal representing noise. The first two of these signals are analyzed because the levels of these two signals vary in response to the type or level of stress a user has experienced.
[0036] Referring back to FIG. 6, the acquired optical signals are pre-processed 605 in order to be able to ascertain various blood flow features from the signals. In one embodiment, pre-processing 605 comprises a filtering leaving the low frequency, large amplitude signal. Large, low frequency signals in the range 0.05-0.12Hz are examined for frequency, magnitude, consistency of variance, relative power and other parameters. FIG. 9 illustrates determining some of these parameters. The intervals between subsequent peaks [ ilsi3 ], and subsequent valleys [ i2 ], can be used to derive
instantaneous and average frequency measures. The height from peak to valley [ hi,h3 ], and valley to peak [ h2,h4 ], may also be used as instantaneous or averaged magnitude measures. There are also techniques for measuring these features on an unfiltered signal. These features represent changes in the cardiovascular system that can be triggered by emotional changes in the body. For example, these may relate to a change in blood pressure, caused by an increase in stress.
[0037] In another embodiment, pre-processing 605 comprises filtering leaving the higher frequency, smaller amplitude signal. Smaller, higher frequency signals in the range 0.5-2Hz are examined for frequency, magnitude, consistency and other parameters. FIG. 10 illustrates determining some of these parameters. The intervals between subsequent peaks [ ii,i3..i9 ], and subsequent valleys [ i2,i4..ii0 ], can be used to derive instantaneous and average frequency measures. The height from peak to valley [ hi,h3..hn ], and valley to peak [ h2,h4..hi0 ], may also be used as instantaneous or averaged magnitude measures. There are also techniques for measuring these features on an unfiltered signal. These features relate to the heartbeat. The size and variance of these peaks may relate to a wide range of cardiovascular parameters that are modulated by emotional changes in the body. Since these peaks relate to blood flow features such as the peak absorption level observed by the sensor, changes in these signals can relate to cardiovascular changes due to emotional states.
[0038] Additionally, preprocessing can include assessing the quality and/or quantity of the collected data. In some embodiments this comprises determining an amount of time or a number of heart beats between interruptions to the data. Data collection can be interrupted by interference in the optical signals due to motion, for example.
[0039] Table 1 lists exemplary blood flow features that can be used to determine a user's stress state. In some embodiments, these blood flow features are determined from the optical signals. Table 1 includes four categories of blood flow features that can be determined from the optical signal. The features in two of these categories, 2 and 4, are features that are not available via electrocardiogram (ECG), a more conventional method used to assess stress states in individuals. These signals relate to peripheral vascular activity that can be indicative of the impacts of emotional states on the body. The features in category 1 can be determined via ECG and thus in other embodiments, if blood flow features from category 1 are used to determine stress states, the input data can be from ECG in addition to or in place of optical data.
Table 1.
Figure imgf000012_0001
Heart rate (HR) Mean & Std Mean and standard deviation of all heart rates within the measurement time window.
HR Mean & Std Norm Mean and standard deviation of all heart rates within the measurement time window with a baseline heart rate subtracted to normalize between subjects.
RMSSD Square root of the mean of the sum of squares of
differences between adjacent RR intervals.
NN50 count Number of RR intervals exceeding 50 ms within the measurement time window. pNN50 NN50 count divided by the total number of all RR
intervals in the measurement time window.
NN20 count Number of RR intervals exceeding 20 ms within the measurement time window. pNN20 NN20 count divided by the total number of all RR
intervals in the measurement time window.
RR Range Maximum RR interval - Minimum RR interval
RR spectral purity Magnitude of the peak frequency of the RR interval pattern, divided by some or all of the magnitudes in adjacent frequencies
Category 2: Time Domain Peak Morphology
Variable Description
Median Heart Rate Frequency The median peak to valley and valley to peak height. Peak height May alternatively use valley to peak height, or both.
Peaks in the 0.3Hz to 3.5Hz range only are considered, specifically those corresponding to frequencies near the heart rate.
Category 3: Frequency Domain Analysis
Variable Description Peak Frequency Peak frequencies of the power spectral density (PSD) estimate for the VLF, LF, and HF frequency bands. [ 3 Variables ]
Very Low Frequency (VLF) Power from ( 0-0.04 ] Hz.
Low Frequency (LF) Power from [ 0.04-0.15 ) Hz.
High Frequency (HF) Power from [ 0.15-0.4 ) Hz
LF Normalized LF power in normalized units: LF/(Total Power- VLF)* 100.
HF Normalized HF power in normalized units: HF/(Total Power- VLF)* 100.
LF/HF Ratio LF/HF
Triangular Index Starting with a histogram of RR intervals, group
intervals into discrete ranges [ 50 - 1000ms each ] and divide the number of entries in the modal range [ i.e. the range with the most number of entries ] by the sum of the number of entries in all the other ranges.
Category 4: Time Domain Analysis
Variables Descriptions
Low Frequency Wave Peak detect over filtered PPG waveform to extract mean period of wave in the 0.05Hz-0.12 Hz frequency range.
Low Frequency Wave Pearson correlation with 0.05Hz-0.12Hz wave Correlation frequency sinusoid. This is the maximum or average value.
Low Frequency Wave Mean peak to peak amplitude of the low-frequency Amplitude [0.05-0.12 ] Hz wave over the epoch. [0040] The identified blood flow features are used to determine 607 information about the user's stress status. In a first embodiment, a pre -trained algorithm is used to convert the observed feature levels to an observed stress type, level or other descriptor. The algorithm is trained in advance using data collected from subjects experiencing the types, levels and other categories of stress to be detected. Alternatively, thresholds for various blood flow features and combinations of each feature's level are used. For example, if a certain proportion of features are beyond a threshold set to indicate stress, the data analyzed could then be considered to represent an elevated stress state.
[0041] Each of the above-identified methods for determining 607 the user's stress status can be normalized for individual users. In one example, resting heart rate is determined for a user after the device is purchased and activated. That resting heart rate is used to normalize the data for the user. Normalizing the process for the user is described in further detail in Example 2.
[0042] In some embodiments, processor 101 determines a quality or quantity of the collected optical data and determines the user's stress state only when the quality and/or quantity of data exceeds a threshold. The thresholds for sufficient quantity or quality of data can be different based on the context of the user as the data is collected. If a user is moving vigorously, fewer of the user's heartbeats may be collected due to interference in the optical signal from motion. Ideally, in order to determine the user's stress state, the processor 101 uses data from the majority of the user's heartbeats in a given period of time (for example 3 or so minutes). However if the processor 101 determines that the user is moving, the processor 101 applies a lower threshold as to what is sufficient data to determine the user's stress state. While the assessment of the stress state may not be as accurate as when the higher threshold is applied, it is more useful to the user to have an assessment of stress state as opposed to no assessment.
[0043] The determined stress state is stored and is displayed to a user via the display 104 in response to a query received via the user interaction points on the face of the device 100. Additionally or alternatively, the stress status is displayed on the display 104 automatically. The status may be displayed automatically if the level of stress determined by the processor 101 exceeds a threshold. Providing the information to the user automatically is useful to alert the user that she is experiencing a high amount of stress. The processor 101 may also provide for display information to assist the user in reducing stress.
[0044] In some embodiments, alternative biological parameters are used in combination with the blood flow features in the determination of an individual's stress state. Table 2 identifies exemplary parameters that can also be used.
Table 2. Alternative Biological Signals
Variables Descriptions
Skin Temperature Skin temperature gradient, minimum, maximum,
average, range, or standard deviation over time. The presence of changes may also be used to signify a skin temperature event, which can then be used as a variable.
Temperature difference Difference between skin and ambient temperature, be it the minimum, maximum, average, range, or standard deviation over time. The presence of changes may also be used to signify a temperature difference event, which can then be used as a variable.
Blood Pressure Blood pressure minimum, maximum, average, range or standard deviation over time. May be estimated from the optical data via low-frequency signal oscillations or pulse speed measurements. The presence of changes may also be used to signify a blood pressure event, which can then be used as a variable.
Electrodermal Activity (EDA) Skin conductance changes over a time window,
including gradient, range, maximum or standard deviation. Patterns such as the ration of the time to rise versus the time to fall.
[0045] In one embodiment data is collected from users experiencing various stress types, levels of stress and stress events, a data set for each stress type to be detected can be prepared. These data sets are then used to train a classifier such as a Support Vector Machine ( SVM ), Random Forest or other machine learning techniques. The classifier can be trained with all or some of the features of Tables 1 and 2. When data is presented to the system from a user, it can be decomposed into the features used to train the classifier which in turn allow for classification of the stress state of the user.
[0046] In an alternative embodiment, determination of a user's stress state may occur at a remote processor instead of on device 100. In such an embodiment, the remote processor implements the functionality of processor 101. The process described in FIG. 6 for determining a user's stress state can be apportioned between processor 101 and the remote processor in various ways. In some embodiments, device 100 transmits raw signal acquired from the optical sensor 407 to the remote processor and all processing is accomplished on the remote processor. In other embodiments, identified features of blood flow are transmitted to the remote processor. Transfer of the process between the device 100 and remote processor can occur at any step in between as well.
[0047] The received stress data and determined stress states can be stored remotely regardless of where the stress states were determined. For example, the information can be stored on a remote server so that the user can access the information via multiple devices such as a laptop computer, tablet computer, smartphone and the like. Guidance for the user on managing her stress states can then be provided to the user on these other devices. This is useful as the display 104 on device 100 has limited space for displaying information.
[0048] Additionally, the remote server may store various other information about the user such as a calendar application. In such an embodiment, the remote server may match the time stamps from the collected data and determined stress state to calendar entries for the user and display the user's stress states during a given time period next to the calendar from that time period. The remote processor may also instruct processor 101 to activate the optical sensing system 203 during times when the calendar application indicates there is an appointment. This is more beneficial to a user than having the stress state only determined at predetermined intervals such as every half hour.
EXAMPLE 1
[0049] In this example the features of Category 1 in Table 1 were used. The smaller feature set was found to be predictive and is thus useful as it reduces processing. When the method is implemented on a wearable device, it beneficially allows for longer data collection on the limited storage in the wearable device.
[0050] In order to develop the algorithm, studies were performed with a cohort of subjects varying in age, gender, background and physical dimensions. During these studies, subjects were subjected to various situations and stimuli designed to elicit particular emotional states. The study was performed with three phases:
1. Phase 1 - Subjects were allowed to relax in a calm setting, with minimal ambient noise and in comfortable seating. They were instructed to enjoy a calm, relaxing time while staying awake.
2. Phase 2 - Subjects engaged in a series of cognitive exercises under time pressure to induce a state of cognitive stress.
3. Phase 3 - Phase 1 was repeated to induce a state of calm.
[0051] The timing of each phase was recorded so that the biometric data from each phase could be identified and labeled with the specific emotional state of the subject. The data recorded in this study includes data from a wearable device such as that described above, as well as other devices.
[0052] Signals that can be used in this experiment include:
1. Blood flow, via an optical sensing system such as that described above
2. Motion, via an accelerometer, pressure sensor or similar sensing modality
3. Skin temperature
4. Core temperature
5. Ambient temperature
6. Galvanic skin response
7. Cortisol levels
8. ECG
9. Ambient humidity
10. Electro-dermal activity (EDA) from a second site (if a wrist- worn device was used, a finger may be used to collect EDA).
Other signals such as blood pressure (11) and respiration rate (12) (as determined from heart beat variance or otherwise) could also be used.
[0053] Some of these signals are sensed continuously, at rates of lHz or faster (1,2,3,4,5,6,8,10) and some are sampled at specific times before, during and after the study (7,9,11). [0054] In the experiment users answered surveys after each phase of the study, indicating their level of different types of emotions including relaxation, cognitive stress, frustration, anxiety and arousal. An initial study including 30 subjects was performed, but subsequently extended with more subjects and a wider range of demographics. In one instance, the signals recorded were: heart beats (via an optical sensor), motion (via an accelerometer), ECG and Cortisol levels.
[0055] In subsequent studies, different stimuli were used and data sets processed in the manner outlined below. In one such subsequent study, the signals recorded were: heart beats (via an optical sensor), motion (via an accelerometer), skin temperature, ambient temperature, galvanic skin response, ECG, and Cortisol levels.
[0056] Data from such a study was used to train a detection algorithm. This algorithm was then used to classify data from other subjects, recorded from their normal lives, into one of the states the algorithm aimed to identify. In one experiment, data from each subject state was labeled (for example in the study described above these labels could reflect a calm set and a stressed set). Data for a particular class from all subjects was combined into a large pool to represent the biological response to each emotional state.
[0057] For each subject in the study, the data collected from each class was segmented into time windows. In one embodiment this window was 3 minutes.
Overlapping segments were used. For example, the first three minutes of the calm phase could be a time window, as could the three minutes between the first and fourth minute. In this way, the second time window overlaps the first by one minute. If an overlapping system is used, the overlap can be as much as one second less than the window size, or as small as one second.
[0058] The data from each time window was labeled with the class it was taken from and decomposed into features such as those in Tables 1 and 2. The duration of the time window for each individual feature may be different.
[0059] In the process of training, or by explicitly analyzing each feature's variance in the two states, it was possible to determine which features are most able to distinguish between the given classes to be detected. All features can be used, however it may also be advantageous only to use a subset that most effectively distinguish the classes or a subset that is most convenient to measure subsequently from device wearers. One embodiment of a classifier architecture that facilitates this feature selection was a Random Forest classifier. This classifier not only trains a model to separate the classes under
consideration, but can also produce metrics as to which features are most important in separating classes. If more than 2 emotional states are to be detected, different features may be most powerful at separating different combinations of classes.
[0060] Other classifiers may be used, such as, but not limited to, Linear Discriminant, Support Vector Machine, Linear Regression or Neural Network. A combination of classifiers may be optimal, since different class combinations may be more optimally separated by different classifier architectures. Different classifier architectures may be best at separating classes with different feature sets. For example, even when classifying the same classes, the features that are optimal for a Random Forest classifier may not be the same as features that perform best if a Linear Discriminant is used.
[0061] Once a classifier has been trained, it is possible to take data measured from a wearable device such as that described above and have the classifier output a score representing the likelihood that the data belongs in one of the classes available for classification. In the same way that time windows were created and decomposed into features to train the classifier, data of the same duration time window is collected and decomposed into features. The features are then used to evaluate the emotional state of the wearer at that time.
[0062] The time window that is used to train the classifier may be different to that used to evaluate a stress level using that classifier. For example, a classifier was trained using 3 minute time windows, but was used to evaluate recordings of just 1 minute duration.
[0063] The likelihood may be represented as a level, for example the degree to which the user is in one state versus another, or as a means to detect events over time. Events may be detected by thresholds that identify a change in likelihood over a period of time. For example, if a stress likelihood score from a range of 0 to 1 were to increase by more than 0.25 over 5 minutes, this could be classified as a stress event.
[0064] It should be noted that, where feature data can be sourced from multiple sensors, a different sensor may be used to train the algorithm than to operate it as a detector. For example, one feature used was Heart Rate Mean & Std, however the algorithm was trained using heart beat data from the ECG and then used as a detector via heart beat information sourced from a wearable optical sensor.
EXAMPLE 2 - NORMALIZATION
[0065] In order to correct for inter- subject differences, data may be used for calibration or normalization. Normalization method parameters may include biasing terms (addition and subtraction), scaling terms (multiplication and division), or other non-linear processing parameters such as raising variables to a given power, and remapping the feature space via logarithmic, exponential or logistical transforms. Normalization parameters may be derived from scientific literature, or dynamically from the data itself using statistical or unsupervised learning techniques. [0066] For example, one feature that may be important in assessing the presence of an emotional arousal state is an increase in heart rate magnitude. Since different subjects may naturally have different heart rate magnitude levels when calm or aroused, it may be necessary to normalize a subject's observed heart rate magnitude. In one embodiment, the user's data is recorded during a 24 hour period and this data is used to generate a biasing and scaling term. A median and standard deviation of heart rate magnitude during relatively inactive periods could be used to normalize for an individual. By subtracting the median from the subject's observed heart rate magnitude values and dividing by the standard deviation, the observed measures of heart rate magnitude may be normalized to similar levels for all subjects.
[0067] In another embodiment, heart rate is scaled against an estimate of the user's maximum heart rate. One estimate for maximum heart rate is 220-(user's age).
[0068] Another feature that may be important in assessing the presence of an emotional arousal state is a decrease in heart rate variance. Since different subjects may naturally have different heart rate variance levels when calm or aroused, it is possible to normalize by subtracting a subject's baseline heart rate variance level. In one
embodiment, the user's data is recorded during sleep and this data is used to generate a baseline. A median of inter-beat intervals during sleep could be used by subtracting this value from observed measures of inter-beat intervals during the day.
[0069] In another embodiment, the biasing and scaling terms may be derived via an unsupervised learning method (such as k-means clustering with k=2). In this embodiment the observed heart rate magnitude will be biased by the average of the centroids given by the unsupervised learning method, and scaled by the magnitude of the centroid distances from each other. [0070] In subsequent use of the trained classifier, a users' data may again be normalized before being processed by a classifier to correct for their personal baseline, maximum, or range.
EXAMPLE 3 - WEIGHTED TRAINING EXAMPLE
[0071] Different subjects respond to stimuli in the data collection studies in different ways. While some stimuli will make one subject emotionally aroused, it may not have the same impact on the emotional state of another subject. In order to correct for this inter- subject variance, it is possible to use objective or subjective measures of actual response to weight the impact of each subject's data for a given class in the development of a classifier for that class.
[0072] For example, in the study described above, the survey results after each phase could be used to weight the time windows during training. The responses to a question asking the level of relaxation could be used as follows:
• For the calm phase, survey responses with higher relaxation scores would be weighted more heavily and contribute more to the classifier training.
• For the stress phase, survey responses with lower relaxation scores would be weighted more heavily and contribute more to the classifier training.
[0073] The same weighted training scheme can be used with Cortisol measurements taken at transitions in the study (for example, at the start of each phase). Individual measurements, or an overall measurement for the study can be used to weight the contribution of data from any one subject. Since Cortisol is a hormone released in response to stress, and since hormone release can take much longer than the duration of a stimulus, measurement over time, after the study is complete, can give more insight into how stressful a study was for a subject. EXAMPLE 4 - ADAPTIVE OPERATION EXAMPLE
[0074] Once a classifier has been trained, its operation can be updated in an ongoing basis to better match a single user's biosignals over time. For example, as baseline measures of cardiac function such as resting heart rate and heart rate variance change, the normalization process that is used to generate more consistent features can also evolve. In this way, the algorithm continues to adapt to a user over time and, thereby, maintains the accuracy of its detection of emotional states over time.
[0075] In one embodiment, the system measures the resting heart rate by estimating it during and around times of sleep. This measurement is then used to normalize a feature based on heart rate, by subtracting the most recent resting heart rate measurement. In this way, while a user's heart rate may change, the system is constantly updating their resting heart rate value and normalizing their heart rate based feature with that most recent measurement of resting heart rate. Not only does this normalize across different subjects, but also across time for a single subject.
EXAMPLE 5 - MOTION MITIGATION EXAMPLE
[0076] The optical signals related to blood flow information include noise introduced by motion of the wearer. In order to mitigate such noise interfering with the detection of heart beats and other information, motion sensing may be used. Example motion sensors include an accelerometer, gyroscope, pressure sensor, compass and magnetometer.
[0077] FIG. 11 shows an example of how such motion mitigation may be used. Each sample from an accelerometer 1101 are evaluated 1102 against filtered version of historical values 1103. These differences are summed 1104 over a whole second and evaluated 1105 against a motion threshold 1106 to determine the level of motion contamination for the last second. This level can be used for the determination of whether the data should be used in further processing or not. In this way, a device can evaluate the data being collected via its sensors to determine if sufficient data has been collected (both in quantity and quality) for the evaluation of the user's stress level.
[0078] In the case that the processing of data to evaluate a stress parameter is performed on a different processor to that contained in the wearable device, this mechanism still facilitates the ability for the device to alert the wearer to the availability of such data, even if the computation is performed after the recorded data is transmitted for processing remotely.
[0079] The type, size and nature of the sensed motion may also be used to
dynamically select between different feature sets or sensors used in the algorithm, or to add context to the output of the system. FIGS. 12 and 13 illustrate examples of dynamic sensor and feature selection. FIG. 12 illustrates activity parameters, such as the level of motion as detected by an accelerometer, evaluated 1201 against parameters indicating physical effort. Recent data is characterized as being related to physical effort 1202 or not 1203. The subsequent analysis of stress is given a different context for each of non- physical stress 1204 and physical stress 1205. An example of this would be the detection that the user has been on a recent run. The processor 101 would operate in "non-physical stress" mode and discount data that might otherwise be considered as an indication of stress because after these data are consistent with ordinary response to physical exertion. Elevated heart rate, for example, would be associated with the physical effort of the run rather than a stress event.
[0080] In some embodiments, when physical exertion is detected the processor 101 does not determine stress. In other embodiments, physical exertion is identified as a type of stress. [0081] FIG. 13 illustrates a second example of how multiple sensors could be used to modify the detection of stress parameters. Skin surface sensors can also be used.
Examples include galvanic skin response sensors, perspiration sensors and sweat constituent (Cortisol, alcohol, adrenalin, glucose, urea, ammonia, lactate) analysis. In FIG. 13, a galvanic skin response sensor 1301 is used to detect a stress event. Upon detection of a stress event, optical sensor data is analyzed to characterize 1302 the stress at that time. Optical sensor data is saved 1303 in a format allowing analysis of parameters upon detection of a galvanic skin response event. The combined output of this algorithm would be both timing information 1304, as provided by the galvanic skin response event, and characterization as provided by the optical sensor data analysis. In this case the galvanic skin response sensor may also be used as an input to the stress characterization algorithm.
[0082] Additional sensors that can be used to provide additional functionality include environmental sensors (e.g., sensors for ultraviolet light, visible light, moisture/humidity, air quality including sensors to detect pollen, dust and other allergens).
ADDITIONAL CONSIDERATIONS
[0083] The disclosed embodiments beneficially allow for monitoring of a user's stress state over extended periods of time because the device collecting the data is worn by the user and is unobtrusive allowing for the device to be worn continually through most daily activities. Using the additional data collected, the system can determine a context for the individuals and thus provide a more personalized assessment of the user's stress. The more personalized assessment includes providing stress information in relation to the user's own baseline as well as using different methods to determine stress based on the user's activity level. [0084] Some portions of above description describe the embodiments in terms of algorithms and symbolic representations of operations on information. These algorithmic descriptions and representations are commonly used by those skilled in the data processing arts to convey the substance of their work effectively to others skilled in the art. These operations, while described functionally, computationally, or logically, are understood to be implemented by computer programs or equivalent electrical circuits, microcode, or the like. Furthermore, it has also proven convenient at times, to refer to these arrangements of operations as modules, without loss of generality. The described operations and their associated modules may be embodied in software, firmware, hardware, or any combinations thereof.
[0085] As used herein any reference to "one embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The
appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.
[0086] Some embodiments may be described using the expression "coupled" and "connected" along with their derivatives. It should be understood that these terms are not intended as synonyms for each other. For example, some embodiments may be described using the term "connected" to indicate that two or more elements are in direct physical or electrical contact with each other. In another example, some embodiments may be described using the term "coupled" to indicate that two or more elements are in direct physical or electrical contact. The term "coupled," however, may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other. The embodiments are not limited in this context. [0087] As used herein, the terms "comprises," "comprising," "includes," "including," "has," "having" or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Further, unless expressly stated to the contrary, "or" refers to an inclusive or and not to an exclusive or. For example, a condition A or B is satisfied by any one of the following: A is true (or present) and B is false (or not present), A is false (or not present) and B is true (or present), and both A and B are true (or present).
[0088] In addition, use of the "a" or "an" are employed to describe elements and components of the embodiments herein. This is done merely for convenience and to give a general sense of the invention. This description should be read to include one or at least one and the singular also includes the plural unless it is obvious that it is meant otherwise.
[0089] Upon reading this disclosure, those of skill in the art will appreciate still additional alternative structural and functional designs for a system and a process for identifying and characterizing stress through the disclosed principles herein. Thus, while particular embodiments and applications have been illustrated and described, it is to be understood that the disclosed embodiments are not limited to the precise construction and components disclosed herein. Various modifications, changes and variations, which will be apparent to those skilled in the art, may be made in the arrangement, operation and details of the method and apparatus disclosed herein without departing from the spirit and scope defined in the appended claims.

Claims

CLAIMS WHAT IS CLAIMED IS:
1. A device for determining stress of an individual, the device comprising:
an optical emitter to emit light into tissue of a user;
an optical sensor to detect an optical signal from the tissue of the user;
a storage communicatively coupled to the optical sensor to store optical signals; a processor communicatively coupled to the storage configured to:
retrieve from the storage optical signals detected over a predetermined period of time;
identify features of blood flow in the tissue based on the retrieved optical signal; and
determine a stress state of the user based on the identified features wherein the stress state comprises a type of stress.
2. The device of claim 1 wherein the determined stress state further comprises a level of stress.
3. The device of any previous claim wherein determining a stress state comprises
identifying a stress event.
4. The device of any previous claim wherein determining a stress state comprises
applying an algorithm to the identified features.
5. The device of any previous claim wherein the algorithm is normalized for the user.
6. The device of any previous claim wherein the optical emitter is configured to emit light having a wavelength between 500 and 600 nanometers (nm).
7. The device of any previous claim wherein the processor is further configured to sample the optical signal at 2Hz-4096Hz.
8. The device of claim 7 wherein the processor is configured to sample the optical signal at 20 Hz-1024 Hz, 30 Hz-1000 Hz, 50 Hz-512 Hz, 64 Hz-512 Hz, 100 Hz-256 Hz or 128 Hz-200 Hz.
9. The device of claim 7 wherein the processor is configured to sample the optical signal at 20, 30, 32, 50, 64, 100, 128, 200, 256, 500, 512, 1000 or 1024 Hz.
10. The device of any previous claim wherein the device further comprises a motion sensor configured to identify motion and determining a stress state comprises determining a stress state based on identified motion.
11. A system for determining stress in an individual, the system comprising a processor configured to:
store a plurality of data comprising optical signals of light transmitted through or reflected from tissue of a user;
retrieve data collected during a predetermined period of time;
identify features of blood flow in the tissue based on the retrieved data; and determine a stress state of the user based on the identified features, the stress state comprising a type of stress.
12. The system of claim 11 wherein one or more features of blood flow are associated with heart rate or heart beat interval.
13. The system of claim 11 or claim 12 wherein determining a stress state comprises applying an algorithm to the identified features.
14. The system of claim 13 wherein the algorithm is normalized for the user.
15. The system of claim 13 wherein the algorithm is normalized based on data collected prior to the predetermined time.
16. The system of claim any one of claims 11-15 wherein the processor is further
configured to provide for display to the user information associated with the determined stress state.
17. The system of any one of claims 11-16 wherein the determined stress state further comprises a level of stress.
18. The system of any one of claims 11-17 wherein determining a stress state comprises identifying a stress event.
19. The system of any one of claims 11-18 wherein the plurality of data further
comprises motion data of the user and determining a stress state is further based on the motion data.
20. The system of any one of claims 11-19 wherein the transmitted or reflected light has a wavelength of between 500 and 600 nm.
PCT/US2013/065965 2012-10-19 2013-10-21 Detection of emotional states WO2014063160A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/436,975 US20150245777A1 (en) 2012-10-19 2013-10-21 Detection of emotional states

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261716405P 2012-10-19 2012-10-19
US61/716,405 2012-10-19

Publications (1)

Publication Number Publication Date
WO2014063160A1 true WO2014063160A1 (en) 2014-04-24

Family

ID=50488824

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/065965 WO2014063160A1 (en) 2012-10-19 2013-10-21 Detection of emotional states

Country Status (2)

Country Link
US (1) US20150245777A1 (en)
WO (1) WO2014063160A1 (en)

Cited By (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9830781B2 (en) 2014-06-13 2017-11-28 Verily Life Sciences Llc Multipurpose contacts for delivering electro-haptic feedback to a wearer
CN107427267A (en) * 2014-12-30 2017-12-01 日东电工株式会社 For the method and apparatus for the state of mind for exporting main body
US10076254B2 (en) 2014-12-16 2018-09-18 Microsoft Technology Licensing, Llc Optical communication with optical sensors
US10092203B2 (en) 2015-08-21 2018-10-09 Verily Life Sciences Llc Using skin resistance measurements to determine timing of bio-telemetry measurements
US10368810B2 (en) 2015-07-14 2019-08-06 Welch Allyn, Inc. Method and apparatus for monitoring a functional capacity of an individual
US10617350B2 (en) 2015-09-14 2020-04-14 Welch Allyn, Inc. Method and apparatus for managing a biological condition
US10791994B2 (en) 2016-08-04 2020-10-06 Welch Allyn, Inc. Method and apparatus for mitigating behavior adverse to a biological condition
US10918340B2 (en) 2015-10-22 2021-02-16 Welch Allyn, Inc. Method and apparatus for detecting a biological condition
US10964421B2 (en) 2015-10-22 2021-03-30 Welch Allyn, Inc. Method and apparatus for delivering a substance to an individual
US10973416B2 (en) 2016-08-02 2021-04-13 Welch Allyn, Inc. Method and apparatus for monitoring biological conditions
US10990888B2 (en) 2015-03-30 2021-04-27 International Business Machines Corporation Cognitive monitoring
US11116397B2 (en) 2015-07-14 2021-09-14 Welch Allyn, Inc. Method and apparatus for managing sensors

Families Citing this family (41)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10251382B2 (en) * 2013-08-21 2019-04-09 Navico Holding As Wearable device for fishing
US9674949B1 (en) 2013-08-27 2017-06-06 Flextronics Ap, Llc Method of making stretchable interconnect using magnet wires
US9554465B1 (en) 2013-08-27 2017-01-24 Flextronics Ap, Llc Stretchable conductor design and methods of making
US10231333B1 (en) 2013-08-27 2019-03-12 Flextronics Ap, Llc. Copper interconnect for PTH components assembly
US9338915B1 (en) 2013-12-09 2016-05-10 Flextronics Ap, Llc Method of attaching electronic module on fabrics by stitching plated through holes
US10015880B1 (en) 2013-12-09 2018-07-03 Multek Technologies Ltd. Rip stop on flex and rigid flex circuits
US9736947B1 (en) * 2013-12-16 2017-08-15 Multek Technologies, Ltd. Nano-copper via fill for enhanced thermal conductivity of plated through-hole via
CN104905803B (en) * 2015-07-01 2018-03-27 京东方科技集团股份有限公司 Wearable electronic and its mood monitoring method
EP3435853B1 (en) * 2016-04-01 2021-09-01 Nitto Denko Corporation A method of deriving systolic blood pressure and/or diastolic blood pressure of a subject
RU2740675C2 (en) * 2016-04-12 2021-01-19 Конинклейке Филипс Н.В. System for increasing a user's sleep efficiency
EP3275362B1 (en) * 2016-07-28 2022-12-28 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Device for monitoring a condition of a living being
JP6826342B2 (en) * 2016-11-29 2021-02-03 東日本旅客鉄道株式会社 Railway comfort evaluation method and railway comfort evaluation device
JP6643748B2 (en) * 2017-07-07 2020-02-12 パナソニックIpマネジメント株式会社 Information providing method, information processing system, and information processing method
WO2019008979A1 (en) * 2017-07-07 2019-01-10 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
CN110582700B (en) * 2017-07-07 2022-07-08 松下知识产权经营株式会社 Information providing method, information processing system, information terminal, and information processing method
CN110573076B (en) * 2017-07-07 2022-09-27 松下知识产权经营株式会社 Information providing method, information processing system, information terminal, and information processing method
WO2019008980A1 (en) * 2017-07-07 2019-01-10 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
JP6924972B2 (en) * 2017-07-07 2021-08-25 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
JP6924964B2 (en) * 2017-07-07 2021-08-25 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
JP6887098B2 (en) * 2017-07-07 2021-06-16 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
WO2019008990A1 (en) * 2017-07-07 2019-01-10 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
JP6924969B2 (en) * 2017-07-07 2021-08-25 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
CN110536638B (en) * 2017-07-07 2022-08-12 松下知识产权经营株式会社 Information providing method, information processing system, information terminal, and information processing method
JP6952281B2 (en) * 2017-07-07 2021-10-20 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
WO2019008995A1 (en) * 2017-07-07 2019-01-10 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
JP6924965B2 (en) * 2017-07-07 2021-08-25 パナソニックIpマネジメント株式会社 Information provision method, information processing system, information terminal, and information processing method
CN110536637B (en) * 2017-07-07 2022-08-12 松下知识产权经营株式会社 Information providing method, information processing system, information terminal, and information processing method
JP6643747B2 (en) * 2017-07-07 2020-02-12 パナソニックIpマネジメント株式会社 Information providing method, information processing system, and information processing method
FR3070591A1 (en) * 2017-09-07 2019-03-08 Ironova METHOD AND MODULE ADAPTED TO IDENTIFY AND QUANTIFY EMOTIONS RELATED BY AT LEAST ONE INDIVIDUAL
US20190133511A1 (en) * 2017-11-09 2019-05-09 Lear Corporation Occupant motion sickness sensing
US10836403B2 (en) 2017-12-04 2020-11-17 Lear Corporation Distractedness sensing system
US10517536B1 (en) * 2018-03-28 2019-12-31 Senstream, Inc. Biometric wearable and EDA method for acquiring biomarkers in perspiration
JP2021519621A (en) * 2018-04-23 2021-08-12 エボニック オペレーションズ ゲーエムベーハー A method for estimating blood pressure and arterial wall sclerosis based on photoplethysmographic (PPG) signals
US10867218B2 (en) * 2018-04-26 2020-12-15 Lear Corporation Biometric sensor fusion to classify vehicle passenger state
US20210228128A1 (en) * 2018-05-08 2021-07-29 Abbott Diabetes Care Inc. Sensing systems and methods for identifying emotional stress events
WO2019216977A1 (en) * 2018-05-08 2019-11-14 Abbott Diabetes Care Inc. Sensing systems and methods for identifying emotional stress events
US10820810B2 (en) * 2018-11-26 2020-11-03 Firstbeat Analytics, Oy Method and a system for determining the maximum heart rate of a user of in a freely performed physical exercise
US11524691B2 (en) 2019-07-29 2022-12-13 Lear Corporation System and method for controlling an interior environmental condition in a vehicle
WO2021147901A1 (en) * 2020-01-20 2021-07-29 北京津发科技股份有限公司 Pressure recognition bracelet
CN111258861A (en) * 2020-02-11 2020-06-09 Oppo广东移动通信有限公司 Signal processing method and related equipment
GB2600126A (en) * 2020-10-21 2022-04-27 August Int Ltd Improvements in or relating to wearable sensor apparatus

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080171922A1 (en) * 2002-10-09 2008-07-17 Eric Teller Method and apparatus for auto journaling of body states and providing derived physiological states utilizing physiological and/or contextual parameter
US7650176B2 (en) * 2000-02-01 2010-01-19 Spo Medical Equipment Ltd. Physiological stress detector device and system
WO2012092221A1 (en) * 2010-12-29 2012-07-05 Basis Science, Inc. Integrated biometric sensing and display device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8099159B2 (en) * 2005-09-14 2012-01-17 Zyto Corp. Methods and devices for analyzing and comparing physiological parameter measurements

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7650176B2 (en) * 2000-02-01 2010-01-19 Spo Medical Equipment Ltd. Physiological stress detector device and system
US20080171922A1 (en) * 2002-10-09 2008-07-17 Eric Teller Method and apparatus for auto journaling of body states and providing derived physiological states utilizing physiological and/or contextual parameter
WO2012092221A1 (en) * 2010-12-29 2012-07-05 Basis Science, Inc. Integrated biometric sensing and display device

Cited By (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10043354B2 (en) 2014-06-13 2018-08-07 Verily Life Sciences Llc Multipurpose contacts for delivering electro-haptic feedback to a wearer
US9830781B2 (en) 2014-06-13 2017-11-28 Verily Life Sciences Llc Multipurpose contacts for delivering electro-haptic feedback to a wearer
US10076254B2 (en) 2014-12-16 2018-09-18 Microsoft Technology Licensing, Llc Optical communication with optical sensors
AU2015372661B2 (en) * 2014-12-30 2021-04-01 Nitto Denko Corporation Method and apparatus for deriving a mental state of a subject
CN107427267A (en) * 2014-12-30 2017-12-01 日东电工株式会社 For the method and apparatus for the state of mind for exporting main body
US11076788B2 (en) 2014-12-30 2021-08-03 Nitto Denko Corporation Method and apparatus for deriving a mental state of a subject
US11120352B2 (en) 2015-03-30 2021-09-14 International Business Machines Corporation Cognitive monitoring
US10990888B2 (en) 2015-03-30 2021-04-27 International Business Machines Corporation Cognitive monitoring
US10368810B2 (en) 2015-07-14 2019-08-06 Welch Allyn, Inc. Method and apparatus for monitoring a functional capacity of an individual
US11116397B2 (en) 2015-07-14 2021-09-14 Welch Allyn, Inc. Method and apparatus for managing sensors
US10092203B2 (en) 2015-08-21 2018-10-09 Verily Life Sciences Llc Using skin resistance measurements to determine timing of bio-telemetry measurements
US10617350B2 (en) 2015-09-14 2020-04-14 Welch Allyn, Inc. Method and apparatus for managing a biological condition
US10964421B2 (en) 2015-10-22 2021-03-30 Welch Allyn, Inc. Method and apparatus for delivering a substance to an individual
US10918340B2 (en) 2015-10-22 2021-02-16 Welch Allyn, Inc. Method and apparatus for detecting a biological condition
US10973416B2 (en) 2016-08-02 2021-04-13 Welch Allyn, Inc. Method and apparatus for monitoring biological conditions
US10791994B2 (en) 2016-08-04 2020-10-06 Welch Allyn, Inc. Method and apparatus for mitigating behavior adverse to a biological condition

Also Published As

Publication number Publication date
US20150245777A1 (en) 2015-09-03

Similar Documents

Publication Publication Date Title
US20150245777A1 (en) Detection of emotional states
US10687757B2 (en) Psychological acute stress measurement using a wireless sensor
US11678838B2 (en) Automated detection of breathing disturbances
CN107106085B (en) Apparatus and method for sleep monitoring
JP5961235B2 (en) Sleep / wake state evaluation method and system
EP2371286B1 (en) Organism fatigue evaluation device and organism fatigue evaluation method
JP7191159B2 (en) Computer program and method of providing subject's emotional state
US9596997B1 (en) Probability-based usage of multiple estimators of a physiological signal
CN109328034B (en) Determining system and method for determining sleep stage of subject
TW201538127A (en) Method and device of sleep detection
JP2021517008A (en) Sleep apnea detection system and method
US20230148961A1 (en) Systems and methods for computationally efficient non-invasive blood quality measurement
Rescio et al. Ambient and wearable system for workers’ stress evaluation
Wang et al. Emotionsense: An adaptive emotion recognition system based on wearable smart devices
EP4033495A1 (en) Activity task evaluating system, and activity task evaluating method
JP2008253727A (en) Monitor device, monitor system and monitoring method
CN115802931A (en) Detecting temperature of a user and assessing physiological symptoms of a respiratory condition
JP2017064364A (en) Biological state estimation apparatus, biological state estimation method, and computer program
Zheng et al. Multi-modal physiological signals based fear of heights analysis in virtual reality scenes
Kuo et al. An EOG-based sleep monitoring system and its application on on-line sleep-stage sensitive light control
Sood et al. Feature extraction for photoplethysmographic signals using pwa: Ppg waveform analyzer
WO2017180617A1 (en) Psychological acute stress measurement using a wireless sensor
Navarro et al. Machine Learning Based Sleep Phase Monitoring using Pulse Oximeter and Accelerometer

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13846468

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 14436975

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 13846468

Country of ref document: EP

Kind code of ref document: A1