Login| Sign Up| Help| Contact|

Patent Searching and Data


Title:
SYSTEM AND METHOD FOR MEASURING ORIENTATION OF AN OBJECT BASED ON IMU MARG ORIENTATION ESTIMATION
Document Type and Number:
WIPO Patent Application WO/2024/023009
Kind Code:
A1
Abstract:
The invention is related to calculate an orientation estimate based on an IMU MARG sensor. It is known to use such combined sensor setup to determine an orientation. From Madgwick it is known to fuse IMU and MARG with only one calculation step per sample by adapting the weight factor based on the rate of orientation change measured by gyroscope and the fusion weighting parameter beta based on the maximum gyroscope measurement error of each axis. The invention addresses the task to find a better solution to this task, especially one that converges fast but does not overshoot and requires less calculation resources. The invention is closely related to Madgwick and amends Madgwick by continuously adjusting the weight factors and using quaternion algebra according to specific scheme.

Inventors:
SEFUNC MERVE (DE)
RADAK HRISTINA (DE)
Application Number:
PCT/EP2023/070432
Publication Date:
February 01, 2024
Filing Date:
July 24, 2023
Export Citation:
Click for automatic bibliography generation   Help
Assignee:
MIMETIK UG (DE)
International Classes:
G01C21/16; G01C25/00
Foreign References:
US20160363460A12016-12-15
US20110307213A12011-12-15
US20160327396A12016-11-10
DE102015111506A12016-04-14
US9529433B22016-12-27
US20160266645A12016-09-15
Other References:
MADGWICK S O H ET AL: "Estimation of IMU and MARG orientation using a gradient descent algorithm", REHABILITATION ROBOTICS (ICORR), 2011 IEEE INTERNATIONAL CONFERENCE ON, IEEE, 29 June 2011 (2011-06-29), pages 1 - 7, XP032318422, ISBN: 978-1-4244-9863-5, DOI: 10.1109/ICORR.2011.5975346
SEBASTIAN O.H.MADGWICK, ANDREW J.L.HARRISON, RAVI VAIDYANATHAN: "Estimation of IMU and MARG orientation using a gradient descent algorithm", IEEE 978-1-4244-9862-8/11, 29 June 2011 (2011-06-29)
MARCEL ADMIRAALSAMUEL WILSONRAVI VAIDYANATHAN: "Improved Formulation of the IMU and MARG Orientation Gradient Descent Algorithm for Motion Tracking in Human-Machine Interfaces", IEEE 978-1-5090-6064-1/17, 2017
DONGPO XUDANILO P. MANDIC: "Quaternion Gradient and Hessian", ARXIV:1406.3587V1 [MATH.NA, 13 June 2014 (2014-06-13)
Attorney, Agent or Firm:
RAFFAY & FLECK (DE)
Download PDF:
Claims:
Claims

1 . A method for IMU sensor data fusion for determining an orientation of an object having attached to it at least one gyroscope (1 ) and at least one accelerometer (1 ) and at least one magnetometer (1 ) featuring: providing an assumed orientation quaternion (qest) and a first initial weight factor (wa,0) and a second initial weight factor (wm,0); providing a sequence of gyroscope data ( w) from such at least one gyroscope showing angular rates as angular rate quaternions, each member corresponding to a point in time; providing a sequence of accelerometer data ( a) form such at least one accelerometer, each member corresponding to a point in time, especially gravitometer data, as accelerometer quaternions and a sequence of magnetometer data (m ) as magnetometer quaternions form such at least one magnetometer, each member corresponding to a point in time; i. performing the following steps, especially using at least one angular, magnetometer and/or accelerometer rate time (rate time) and/or at least one angular, magnetometer and/or accelerometer rate time index (rate time index, k), especially initialized with a point in time or corresponding index calculating quaternion derivative of the angular rate, especially from an angular rate quaternion corresponding to the rate time and/or rate time index; integrating quaternion derivative of the angular rate to update the assumed orientation quaternion, especially by mixing the assumed orientation quaternion with integrated quaternion derivative of the angular rate; determining an updated assumed orientation quaternion by solving the optimization problem to align the assumed orientation quaternion with accelerometer quaternions and/or magnetometer quaternions by the following dual quaternion gradient descent method comprising: characterised by ii. if (B) angular rate and/or angular rate quaternion length is not above a predefined threshold: increasing the first weight factor or setting it to a predefined high accelerometer value if the last accelerometer step reduced the value of an accelerometer cost function defined based on the difference between the assumed orientation quaternion after the later of the three of step i, step iv and step v, especially later of the two of step iv and step v, on the one hand and accelerometer quaternion from the sequence of accelerometer data, especially corresponding to a moment in time similar to rate time and/or rate time index on the other hand and decreasing it if the value of the accelerometer cost function was increased by the last accelerometer step; iii. adapting the second weight factor by increasing it or setting it to a predefined high magnetometer value if the last magnetometer step reduced the value of a magnetometer cost function defined based on the difference between the assumed orientation quaternion after the last magnetometer step or the assumed orientation quaternion after the later of the three, step i, step iv and step v, specially later of the two of step iv and step v, on the one hand and corresponding magnetometer quaternion from the sequence of magnetometer data, especially corresponding to a moment in time similar to rate time and/or rate time index on the other hand and decreasing it if the value of the magnetometer cost function was increased; iv. if (B) angular rate and/or angular rate quaternion length is not above a predefined threshold performing one accelerometer step with the first weight factor for an accelerometer quaternion from the sequence of accelerometer data, especially corresponding to a moment in time similar to rate time and/or rate time index, and one magnetometer step with the second weight factor for the an magnetometer quaternion from the sequence of magnetometer data, especially corresponding to a moment in time similar to rate time and/or angular rate time index, to update the assumed orientation quaternion, preferably by mixing one or more gradients or field quaternion with the assumed orientation quaternion and using the result as the assumed orientation quaternion; v. or (A) performing one magnetometer step with the second weight factor for a magnetometer quaternion from the sequence of magnetometer data, especially corresponding to a moment in time similar to rate time and/or rate time index, to update the assumed orientation quaternion, preferably by obtaining a gradient or field quaternion and mixing it with the assumed orientation quaternion and using the result as the assumed orientation quaternion; vi. increasing the at least one accelerometer rate time and/or rate time index vii. repeating steps i to vi, especially at least more than two times. 2. The method according to claim 1 wherein the sequence of gyroscope data is based on samples of a, especially one three-dimensional, gyroscope, especially after aggregation and/or filtering, wherein preferably the samples include three angles of rotation per sample interval and/or wherein the sequence of magnetometer data is based on samples of a, especially one three-dimensional, magnetometer, especially after aggregation and/or filtering, wherein preferably the samples include three values per sample interval and/or wherein the sequence of accelerometer data is based on samples of a, especially one three-dimensional, accelerometer, especially gravitometer, especially after aggregation and/or filtering, wherein preferably the samples include three values per sample interval.

3. The method according to any of the preceding claims including sampling of the sequence of gyroscope data from a three-dimensional gyroscope and the sequence of accelerometer data from a three-dimensional accelerometer, especially gravitometer, and the sequence of magnetometer data from a three-dimensional magnetometer. 4. The method according to any of the preceding claims 2 or 3, wherein the magnetometer, gyroscope and accelerometer are arranged in a fixed spatial arrangement. 5. The method according to any one of the preceding claims 2 to 4, wherein the magnetometer, gyroscope and accelerometer are attached to a glove or an attachment to a glove, wherein the attachment or glove features 5 to 15, sensor triples, each consisting of a magnetometer, gyroscope and accelerometer. 6. The method according to any of the preceding claims, wherein the sequence of gyroscope data, the sequence of accelerometer and the sequence of magnetometer data is provided wirelessly, especially via Wi-Fi, especially from a glove or glove attachment according to claim 5, wherein especially such data is only provided if movement of the glove and/or sensors is greater than a predetermined movement threshold and/or angular rate quaternion length is above a predetermined transmission threshold. 7. The method according to any of the preceding claims wherein in step iii adapting the first weight factor by setting it to zero if angular rate and/or angular rate quaternion length is equal or above a predefined threshold. 8. The method according to any of the preceding claims wherein repeatedly the updated assumed orientation quaternion is determined for a multitude of sensor triple data or sensor triples. 9. The method according to any of the preceding claims wherein the object is a clothing or a body part.

10. The method according to any of the preceding claims wherein providing the assumed orientation quaternion and a first initial weight factor (wa,0) and a second initial weight factor (wm,0) is done by providing an arbitrary orientation quaternion and/or an arbitrary first weight factor and/or an arbitrary second initial weight factor and/or by providing an orientation quaternion derived from other data, information and/or measurement and/or one as first initial weight factor (wa,0) and one as second initial weight factor (wm,0). 11 . The method according to any of the preceding claims wherein the sequence of gyroscope data is a sequence over time, especially providing for a first multitude of points in time for each point in time gyroscope data and/or the sequence of accelerometer data is a sequence over time, especially providing for a second multitude of points in time for each point in time accelerometer data and/or the sequence of magnetometer data is a sequence over time, especially providing for a third multitude of points in time for each point in time magnetometer data, wherein especially first, second and/or third multitude include the same number of points and/or are identical. 12. The method according to any of the preceding claims wherein performing an accelerometer step is performed as known from Madgwick but using quaternion algebra and/or wherein performing an magnetometer step is performed as known from Madgwick but using quaternion algebra and/or a two-step gradient descent is used to minimize the difference between a true orientation of the object and the assumed orientation estimate and using the accelerometer and magnetometer data to correct the assumed orientation estimate wherein two quaternions are used as a gradient or to build a gradient, and/or a accelerometer step is performed by using one, especially normalized, quaternion derivative derived from an accelerometer quaternion of the sequence of accelerometer data multiplied with the first weight factor (wa,0) to change the assumed orientation estimate, especially by mixing assumed orientation estimate with result of multiplication of quaternion derivative derived from an accelerometer quaternion of the sequence of accelerometer data multiplied with the first weight factor (wa,0), and/or a magnetometer step is performed by using one, especially normalized, quaternion derivative derived from a magnetometer quaternion of the sequence of magnetometer data multiplied with the second weight factor (wm,0) to change the assumed orientation estimate, especially by mixing assumed orientation estimate with result of multiplication of quaternion derivative derived from a magnetometer quaternion of the sequence of magnetometer data multiplied with the second weight factor (wm.o), and/or wherein quaternion derivative derived from a magnetometer quaternion of the sequence of magnetometer data multiplied with the second weight factor (wm,0) and quaternion derivative derived from an accelerometer quaternion of the sequence of accelerometer data multiplied with the first weight factor (wa,0) are mixed to obtain the field quaternion which is mixed with the assumed orientation quaternion to obtain a result used as the assumed orientation quaternion. 13. The method according to any of the preceding claims wherein performing an accelerometer step is performed by a gradient descent method following the step in the direction of the gradient obtained from the sequence of accelerometer data , which is analytically derived following the rules of the quaternion algebra also referred to as HR calculus, and especially normalised, especially minimise the difference between the true orientation and the assumed orientation estimate obtained from the accelerometer data and/or wherein performing a magnetometer step is performed by a gradient descent method following the step in the direction of the gradient obtained from the sequence of magnetometer data, which is analytically derived following the rules of the quaternion algebra, and especially normalised, especially to minimise the difference between the true orientation and the assumed orientation estimate obtained from the magnetometer data.

14. The method according to any of the preceding claims wherein mixing is performed by summing up the objects to be mixed, while each object may be multiplied with an, especially individual, weighting factor prior to summing and/or the result may be multiplied with an factor, especially lower than one.

Description:
System and method for measuring orientation of an object based on IMU MARG orientation estimation

The invention is related to calculating an orientation estimate based on an IMU MARG sensor. In this context "IMU" means inertial measurement unit and "MARG" means magnetic angular rate and gravity. IMU MARG sensors hence are combined sensors or sensor setups that use IMU as well as MARG measuring principles.

It is known to use such a combined sensor setup to determine an orientation of an object. Such sensors may be attached to an object e.g., such as a body part, especially a limb, especially by use of clothing, e.g., glove. Such a sensor may be attached to or integrated into such clothing. By use of such sensors, the orientation and/or movement of the object may be estimated and/or tracked.

It is known from US 201 1 /0307213 A1 and US 2016/0327396 A1 to use quaternion-variables for IMU and MARG fusion by Kalman filters or by complicated comparisons and decisions regarding fusion. Gloves featuring such sensors are known from DE 10 2015 1 1 1 506 A1 , US 9,529.433 B2 and US 201 6/0266645 A1 .

From "Estimation of IMU and MARG orientation using a gradient descent algorithm", Sebastian O.H. Madgwick, Andrew J.L. Harrison, Ravi Vaidyanathan, IEEE 978-1 -4244-9862-8/1 1 , Rehab Week Zurich, ETH Zurich Science City, Switzerland, June 29 - July 1 , 201 1 , referred to by Madgwick, it is known to fuse IMU and MARG with only one calculation step per sample by adapting a weight factor / based on a rate of orientation change measured by a gyroscope and the fusion weighting parameter beta based on the maximum gyroscope measurement error of each axis. The reference further mentions application of such in aerospace, robotics, navigation, human motion analysis, machine interaction and rehabilitation.

"Formulation of a new gradient descent MARG orientation algorithm: Case study on robot teleoperation", Samuel Wilson, Henry Eberle, Yoshikatsu Hayashi, Sebastian O.H. Madgwick, (in the following Wilson) discusses the problem of a two gradient descent method when gradients are not decoupled. The reference also discusses means for decoupling gradients by amending gradients prior to using gradients for a descent and showing that the first 0.2 s are sufficient to classify a gesture.

From "Improved Formulation of the IMU and MARG Orientation Gradient Descent Algorithm for Motion Tracking in Human-Machine Interfaces", Marcel Admiraal, Samuel Wilson, Ravi Vaidyanathan, IEEE 978-1 -5090-6064-1 /17, 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI 2017) November 16 - 18, 2017, Daegu, Korea it is known to optimise the calculation of the gradient to find a direction of a steepest descent for the two-step gradient descent method based on complex algebra and setting weight factor alpha to a fixed value in a specified range.

From "Quaternion Gradient and Hessian", Dongpo Xu, Danilo P. Mandic, arXiv:1406.3587v1 [math. NA] 13 Jun 2014, (in the following [Mandic]) it is known to calculate quaternion derivatives.

The present invention addresses the task of finding a better solution to motion tracking, especially one that converges quickly but does not overshoot and requires fewer calculation resources. The invention is closely related to Madgwick, which is hereby included by reference, but does develop Madgwick's method further to improve convergence and overshoot while attempting to reach a convergence equal to or greater than the physical rate of change of orientation. Madgwick teaches the use of a two- step gradient descent to minimize the difference between a true orientation of the object and an assumed orientation estimate of the object. Madgwick uses accelerometer and magnetometer data to correct the assumed orientation estimate, wherein an accelerometer step is performed by using one, especially normalized, gradient derived from accelerometer data multiplied with a first weight factor to change the assumed orientation estimate by summing assumed orientation estimate with the result of a multiplication gradient derived from accelerometer data with the first weight factor. A magnetometer step is performed by using one, especially normalized, gradient derived from magnetometer data multiplied with a second weight factor to change the assumed orientation estimate by summing the assumed orientation estimate with a result of multiplication of gradient derived from magnetometer data multiplied with the second weight factor. The results of multiplication are mixed to obtain a field gradient which is mixed with the assumed orientation to obtain a result used as assumed orientation. Madgwick uses real numbers.

In this description, if data does not specifically define which data is referred to, preferably gyroscope data, accelerometer data and/or magnetometer data is to be understood.

In this description, if sequence does not specifically define which sequence is referred to, preferably gyroscope sequence, accelerometer sequence and/or magnetometer sequence, even more preferably gyroscope sequence data, accelerometer sequence data and/or magnetometer sequence data, is to be understood.

Orientation estimation is solved by a method for IMU sensor data fusion for determining an orientation of an object having attached to it at least one gyroscope and at least one accelerometer and at least one magnetometer. The present method features: providing an assumed orientation quaternion (which may be chosen arbitrarily, e.g., (1 ,0,0,0) and a first initial weight factor and a second initial weight factor (which may be chosen arbitrarily, e.g., as 1 ); providing a sequence of gyroscope data showing angular rates (Sw) as angular rate quaternions; and providing a sequence of accelerometer data, especially gravitometer data, as accelerometer quaternions and a sequence of magnetometer data as magnetometer quaternions.

The gyroscope data, accelerometer data, and magnetometer data preferably are obtained from the at least one gyroscope and at least one accelerometer and at least one magnetometer, respectively. The sequences of gyroscope data, accelerometer data, and/or magnetometer data may maybe such in time or simply indexed once. The index may correspond to a time, especially evolving time with an index count. The sequences may all be with the same index, time span and/or time spacing or with different index, time span and/or time spacing. Preferably the sequences do show similar or identical time spacing and/or a same number of members of the sequence. The sequences (gyroscope sequence, accelerometer sequence and/or magnetometer sequence) are especially sequences obtained by repeated sampling of the corresponding sensor, especially all using the same sampling frequency/sampling period and/or corresponding to a same timeframe, but. But the sequences do not have to be taken at the exact same times. i. The method further features, calculating a quaternion derivative of the angular rate, describing a rate of change of an earth frame relative to the sensor frame, especially from an angular rate quaternion corresponding to a first point in time and/or by integrating quaternion derivative of the angular rate. A first point in time may be chosen arbitrarily, e.g., as a time corresponding to a start of any of the above sequences (gyroscope sequence, accelerometer sequence and/or magnetometer sequence).

Such motion tracking may be performed e.g., like the notation of Madgwick but using quaternion algebra, called as well HR calculus, e.g., like and/or by integrating a quaternion derivative of the angular rate which may be done by multiplication of the derivative with a sampling period. (In contrast to a first impression, the integration of a derivative does not cancel as the calculation does not use linear algebra and is not commutative, especially when using above formula.) Preferably, the integrating quaternion derivative is used to update the assumed orientation quaternion, especially by mixing the assumed orientation quaternion with the integrated quaternion derivative of the angular rate. To perform such an estimation, in the case that the assumed orientation quaternion which is used is not an (otherwise) assumed orientation but rather is one derived by use of accelerometer and/or magnetometer data, especially field and/or gradient derived therefrom (preferably corresponding to time/index is used that is, similar and/or, especially directly, prior to the time/index to which the magnetometer and/or accelerometer quaternion correspond that will be used to obtain assumed orientation quaternion based on integrated quaternion derivative of the angular rate), e.g., the quaternion derivative of the angular rate is preferably obtained from the quaternion ( s ω) from the previous or similar iteration step or previous or similar iteration time or previous time instance compared to the magnetometer and/or accelerometer quaternion that will be used to obtain assumed orientation quaternion based on integrated quaternion derivative of the angular rate. Therefore, preferably, for obtaining quaternion derivative of the angular rate, an angular rate quaternion corresponding to a point in time similar or earlier, especially one sample interval prior, to the data used to derive the next gradient and/or field and/or to a point in time identical and/or similar to the data used to derive the next gradient and/or field is used. Such a method enhances convergence. The present method further features repeatedly and/or continuously determining an updated assumed orientation quaternion (the updated assumed orientation quaternion being preferably performed in various ways) by solving the optimization problem to align (by updating) assumed orientation quaternion with accelerometer quaternions and magnetometer quaternions by a dual gradient descent method. The dual gradient descent method which may be described as involving the performance of the following steps, especially using an angular, magnetometer and/or accelerometer rate time (rate time) or angular, magnetometer and/or accelerometer rate time index (rate time index) initialized with a point in time similar, identical, or later than the first point in time or similar, identical, or higher than its index. The method features:

Preferably performing one first accelerometer and/or magnetometer step for the first accelerometer quaternion and/or magnetometer quaternion as known from Madgwick but using quaternion algebra.

Such a first accelerometer step is constituted by one first accelerometer step with the first initial weight factor for the first accelerometer quaternion and/or such first magnetometer step is constituted by a first magnetometer step with the second initial weight factor for the first magnetometer quaternion as known from Madgwick. This first step is especially performed to determine an updated assumed orientation quaternion by solving the optimization problem to align assumed orientation quaternion with accelerometer quaternions and/or magnetometer quaternions. For the alignment assumed orientation quaternion is changed, especially by mixing with at least one gradient or field quaternion. Performing one first accelerometer step may include performing one first accelerometer step with a weight factor being the first initial weight factor (w a,0 ) and a first accelerometer quaternion from the sequence of accelerometer data, especially corresponding to a moment in time similar to and/or later than the angular, magnetometer and/or accelerometer rate time or the angular, magnetometer and/or accelerometer rate time index, and/or one first magnetometer step with a second weight factor being the second initial weight factor (wm,O) for a first magnetometer quaternion from the sequence of magnetometer data, especially corresponding to a moment in time similar to and/or later than the angular, magnetometer and/or accelerometer rate time or the angular, magnetometer and/or accelerometer rate time index.

Similarity is, in general, preferably assumed if the difference in time is not larger than three times, especially 1 .5, times, the difference in time between members of the corresponding sequence and/or any of the sequences, especially equal to and/or smaller than the difference in time between members of the corresponding sequence and/or any of the sequences. The step is especially performed by mixing using the weight factor, especially to multiply with the corresponding gradient.

The invention amends Madgwick further by continuously adjusting the weight factors, for all or further steps.

This continuously adjusting of the weight factors may be archived by: ii. if the angular rate and/or the angular rate quaternion length is not above a predefined threshold, increasing the first weight factor or setting the first weight factor to a predefined high value if a last accelerometer step reduced the value of an accelerometer cost function and decreasing it if the value of the accelerometer cost function was increased by the last accelerometer step, especially between assumed orientation quaternion after last accelerometer step or assumed orientation quaternion after the later of the three, step i, step iv and step v, and last next or the next accelerometer quaternion on the other hand and/or accelerometer quaternion from the sequence of accelerometer data, especially corresponding to a moment in time similar to and/or prior to the angular, magnetometer and/or accelerometer rate time or angular, magnetometer and/or accelerometer rate time index and/or, especially directly, prior to last accelerometer step or assumed orientation quaternion, especially directly, prior to the later of the three, step i, step iv and step v on the other hand. The accelerometer cost function is defined based on the difference between the assumed orientation quaternion after the last accelerometer step or the assumed orientation quaternion after a later of the three, step i, step iv and step v, on the one hand and last next or the next accelerometer quaternion on the other hand and/or accelerometer quaternion from the sequence of accelerometer data, especially corresponding to a moment in time similar to and/or prior to the angular, magnetometer and/or accelerometer rate time or angular, magnetometer and/or accelerometer rate time index and/or, especially directly, prior to last accelerometer step or assumed orientation quaternion, especially directly, prior to later of three step i, step iv and step v on the other hand. iii. especially for a next magnetometer quaternion, adapting the second weight factor by increasing it or setting the second weight factor to a predefined high value if the last magnetometer step reduced the value of a magnetometer cost function and decreasing it if the value of the magnetometer cost function was increased. The magnetometer cost function is defined based on the difference between assumed orientation quaternion after the last magnetometer step or the assumed orientation quaternion after the later of the three of step i, step iv and step v, on the hand and last next or the next magnetometer quaternion and/or corresponding magnetometer quaternion from the sequence of magnetometer data, especially corresponding to a moment in time similar to and/or prior to the angular, magnetometer and/or accelerometer rate time or angular, magnetometer and/or accelerometer rate time index and/or, especially directly, prior to last accelerometer step or assumed orientation quaternion, especially directly, prior to later of the three of step i, step iv and step v on the other hand iv. if the angular rate and/or the angular rate quaternion length is not above a predefined threshold, performing one accelerometer step with the first weight factor for an accelerometer quaternion and/or a next accelerometer quaternion from the sequence of accelerometer data, especially corresponding to a moment in time similar to and/or later than the angular, magnetometer and/or accelerometer rate time or angular, magnetometer and/or accelerometer rate time index, and one magnetometer step with the second weight factor for an and/or the next magnetometer quaternion from the sequence of magnetometer data, especially corresponding to a moment in time similar to and/or later than angular, magnetometer and/or accelerometer rate time or angular, magnetometer and/or accelerometer rate time index, especially by obtaining a gradient and/or field quaternion and mixing the gradient and/or field quaternion with the assumed orientation quaternion and using the result as a new/updated assumed orientation quaternion v. else (meaning that the angular rate and/or the angular rate quaternion length is above the predefined threshold) performing one magnetometer step with the second weight factor for an and/or the next magnetometer quaternion from the sequence of magnetometer data, especially corresponding to a moment in time similar to and/or later than angular, magnetometer and/or accelerometer rate time or angular, magnetometer and/or accelerometer rate time index, especially by obtaining a gradient or field quaternion and mixing the gradient or field quaternion with the assumed orientation quaternion and/or the integrated quaternion derivative of the angular rate and using the result as a (new/updated) assumed orientation quaternion vi. preferably calculating a quaternion derivative of the angular rate, especially from an angular rate quaternion corresponding to the angular, magnetometer and/or accelerometer rate time or a next or angular, magnetometer and/or accelerometer rate time index or the next; vii. integrating the quaternion derivative of the angular rate to update the assumed orientation quaternion, especially by mixing an integrated quaternion derivative of the angular rate with the assumed orientation quaternion; viii. increasing the angular, magnetometer and/or accelerometer rate time or angular, magnetometer and/or accelerometer rate time index; and ix. repeating steps i to v and, especially step viii, or steps ii to viii, or steps i to viii, especially at least more than two, especially more than five, times, especially by going through the sequences.

Preferably each member/quaternion of any sequence is only used once. Preferably the method is performed going forward through a sequence, especially through all of the sequences.

First weight factors, as well as predefined high values, may differ for a first and a second weight or they may be identical. The first weight factors may be chosen freely but preferably are chosen high, e.g., in the range from 1 to 10, especially relative to the weight of the integrated quaternion derivative of the angular rate.

A three-axis gyroscope may be used to measure the angular rate about x, y and z axes of a sensor frame, and the angular rates are termed ω x , ω y and ω z , respectively. These parameters (in rads - 1 ) may be arranged into a vector as follows:

The quaternion derivative describing rate of change of the earth frame relative to the sensor frame may be calculated as:

The operate denotes a quaternion product and the accent denotes a normalized quaternion of unit length.

Advantageously, the present method goes through a sequence of magnetometer data and a sequence of accelerometer data. In each repetition a next of the sequence may be used, and is denoted as the "next". The sequence prior to that the next, that especially was addressed prior to handling the next is then referred to as the "last next". Such a sequence may be done by a running index of each sequence. For easy reading, the description uses one common index for both sequences, e.g., k. k may be incremented for obtaining the next which is then addressed by k. After such increment by one the last next may be addressed by k-1 .

Increasing of weight factors may be done by setting the weight factor to a high value if not already set to such high value. Such high value may be predefined, e.g., as 2.0 or a value in the range of 1 to 10. Decreasing of weight factors may be done by dividing the weight factor by a factor e.g., 2 or in the range of 1 .5 to 8. This keeps the method easy and fast and has proven to be sufficient. As an alternative, e.g., the Armijo rule may be used to increase and decrease the weighting factors.

The cost functions may be defined as:

If <= 0 increase step size for the accelerometer.

If | <= 0 increase step size for the magmctomctcr.

Using the notation f a is the cost function for the accelerometer q is the quaternion estimate from last iteration g is the gravity vector in quaternion form g = (0, 0, — 1, 0) a is the accelerometer sample after applying filter and in quaternion form a = (0, a x , a y , a z ) is a guternion product operator f m is the cost function for the magnetometer h' is an equivalent reference vector for magnetometer calculated according to the Eq. (7) from Wilson m' is an equivalent magnetometer measurement calculated according to the Eq. (6) from Wilson

Eq. (7) from Wilson applied here means h' = | h x g | , g representing the direction of gravity e.g., as h representing the direction of magnetic north, e.g., as where Vrx and Vrz represent the horizontal and vertical component of the local geomagnetic field.

Eq. (6) from Wilson applied here means m'k = | m k x a k | .

As (one dimensional) cost functions as well Fa and Fm may be used as

The cost functions may be based on and, especially given by, the difference between the assumed orientation quaternion after the later of the three of step i, v and iv, on one hand and the next accelerometer/magnetometer quaternion on the other hand, meaning especially the current accelerometer/magnetometer selected one from the sequence.

The assumed orientation quaternion after the later of the three of step i, step iv and step v, especially means the assumed orientation quaternion after step i, if no step iv or v was performed so far, and after one step iv or v was performed, or the assumed orientation quaternion after the latest of step iv or v.

Advantageously, the cost functions are given by the claimed difference on which they are based respectively.

In some applications it is preferred to adapt the first weight factor in step ii by setting it to zero if the angular rate and/or the angular rate quaternion length is equal to or above a predefined threshold. Providing the assumed orientation quaternion means provision of an initial assumed orientation quaternion. This this may e.g., be an arbitrary quaternion, preferably (1 ,0, 0,0) or a quaternion derived from any other data like measurements, historical values or the like, e.g., one may derive the assumed orientation quaternion from any available accelerometer and/or magnetometer measurement(s).

The mixing may as well be described as combining. It is preferably performed by use of the weighting factors. In this description the weighting factor of the integrated quaternion derivative of the angular rate is assumed to be 1 . But it will be understood that the weighting factors may be scaled as desired, especially as it is preferred to include a normalisation of the assumed orientation quaternion. Mixing of two Quaternions Q1 and Q2 may be described by:

Wa * Q1 + Wb * Q2 with Wa and Wb being weighting factors.

A field quaternion is a quaternion describing an orientation and is preferably calculated only from the accelerometer and magnetometer measurements.

In this description, the test of whether a threshold of movement is reached is performed by a < = and > but may as well be amended to < and > = .

Preferably, the sequence of gyroscope data is based on samples of a, especially three-dimensional, gyroscope, especially after aggregation and/or filtering, wherein preferably the samples include three angles of rotation per sample interval.

Advantageously, the sequence of magnetometer data is based on samples of a, especially three-dimensional, magnetometer, especially after aggregation and/or filtering, wherein preferably the samples include three values per sample interval.

Preferably, the sequence of accelerometer data is based on samples of a, especially three-dimensional, accelerometer, especially a gravitometer, especially after aggregation and/or filtering, wherein preferably the samples include three values per sample interval.

Advantageously, the method includes sampling of the sequence of gyroscope data from a three-dimensional gyroscope and the sequence of accelerometer data from a three-dimensional accelerometer, especially a gravitometer, and the sequence of magnetic field data from a three-dimensional magnetometer.

Each sequence of accelerometer data, sequence of gyroscope data from and/or sequence of magnetometer data preferably is a sequence in time, e.g., giving accelerometer/magnetometer/gyroscope measurements for concurrent points in time and/or indexed by angular, magnetometer and/or accelerometer rate time and/or angular, magnetometer and/or accelerometer rate time index. The points in time of those sequences may be the same, similar, or different for each sequence. E.g., every 1 ms a measurement of accelerometer, magnetometer, gyroscope is performed simultaneously, thereby building sequences with the same points in time. Therefore, the sequences may be identical, similar, or different in regard to the time aspect. But while every 1 ms a measurement of accelerometer, magnetometer, gyroscope is performed the measurements may not be performed simultaneously, thereby leading to sequences covering approximately the same time span, having the same time spacings and the same number of members but not being identical to one another. Such sequences may be used as sequences to perform the method with nearly no downside. Such points in time, like every consecutive ms in the respective time span, may be referred to by a, preferably for all sequences, a common time index, e.g., angular, magnetometer and/or accelerometer rate time or angular, magnetometer and/or accelerometer rate time index.

Preferably the magnetometer, gyroscope and accelerometer are arranged in a fixed spatial arrangement relative to one another. This provides the most reliable and consistent measurement and calculation.

Even more preferably, the magnetometer, gyroscope and accelerometer are attached to a glove or are an attachment to a glove or an attachment to a body part, especially a person's limb. With great avail, the attachment or glove features 5 to 15 sensor triples, each triple including a magnetometer, a gyroscope and an accelerometer, especially all measuring three dimensionally. This configuration on a glove, for example, enables detection of movement of a person's fingers and palm, each as an estimated orientation quaternion. The invention is especially useful when attached to a body part, either directly or indirectly by connection to clothes as body parts do move in an unknown scheme and such movement may be of interest in different fields of technique, e.g., to steer any physical and/or virtual object by movement of a body part.

Preferably providing an assumed orientation quaternion and a first initial weight factor and a second initial weight factor is done by providing an arbitrary orientation quaternion and/or an arbitrary first weight factor and/or an arbitrary second initial weight factor and/or by providing an orientation quaternion derived from other data, information and/or measurement and/or one as first initial weight factor and one as second initial weight factor. Initialisation may be done by use of arbitrary data as the method will move towards the correct orientation. But such may be enhanced by using any other data to set an assumed orientation quaternion to something more fitting to the real orientation as an arbitrary value. Further, weight factors may be initialised using information on speed of movement and/or an assumed error regarding initial assumed orientation.

The sequence of gyroscope data is a sequence over time, especially providing for a first multitude of points in time gyroscope data for each point in time and/or the sequence of accelerometer data is a sequence over time, especially providing accelerometer data for a second multitude of points in time for each point in time and/or the sequence of magnetometer data is a sequence over time, especially providing magnetometer data for a third multitude of points in time for each point in time, wherein especially first, second and/or third multitude include the same number of points, cover approximately the same or the same time span and/or are identical. Such leads to especially good results when attempting to establish a real-world orientation of an object by the present method. The present method may be performed live, e.g., by the sequences not been established when the method is started but being established and fed during performing the method. This is especially advantageous as the method is well suited to be performed live, especially with the speed of measurement, meaning that calculation of one repetition, e.g., of steps i to v or to viii does not take more time as between sampling of two members of each sequence (sampling period), especially not more than 200 ms, especially not more than 10 ms. The method may also be performed on saved data from the past.

It is preferred to perform an accelerometer step as known from Madgwick but using quaternion algebra and/or wherein to perform a magnetometer step as known from Madgwick but using quaternion algebra.

Especially a two-step gradient descent is used to minimize the difference between a true orientation of the object and the assumed orientation estimate and using the accelerometer and magnetometer data to correct the assumed orientation estimate. In the present method two quaternions are used as a gradient or to build a gradient/field, and/or a accelerometer step is performed by using one, especially normalized, quaternion derivative derived from an accelerometer quaternion of the sequence of accelerometer data multiplied with the first weight factor to change the assumed orientation estimate, especially by summing assumed orientation estimate with result of multiplication of quaternion derivative derived from an accelerometer quaternion of the sequence of accelerometer data multiplied with the first weight factor, and/or a magnetometer step is performed by using one, especially normalized, quaternion derivative derived from a magnetometer quaternion of the sequence of magnetometer data multiplied with the second weight factor to change the assumed orientation estimate, especially by summing assumed orientation estimate with result of multiplication of quaternion derivative derived from a magnetometer quaternion of the sequence of magnetometer data multiplied with the second weight factor.

Preferably quaternion derivative derived from a magnetometer quaternion of the sequence of magnetometer data multiplied with the second weight factor and quaternion derivative derived from an accelerometer quaternion of the sequence of accelerometer data multiplied with the first weight factor are mixed to obtain the field quaternion which is mixed with the assumed orientation quaternion to obtain a result which is used as a new/updated assumed orientation quaternion.

Especially the assumed orientation estimate is found by integrating the gyroscope data, e.g., quaternion derivative of the gyroscope quaternion (e.g., and/or is calculated by use of the assumed orientation estimate prior to increasing rate time and/or rate time index, preferably by multiplication with gyroscope quaternion, especially corresponding to the rate time and/or rate time index, describing rate of change of the earth frame relative to the sensor frame), e.g., by multiplication with the sampling period.

Preferably an accelerometer step is performed by a gradient descent method following the step in the direction of the gradient obtained from the sequence of accelerometer data, which is analytically derived following the rules of the quaternion algebra also referred to as HR calculus, and is especially normalised, to minimise the difference between the true orientation and the assumed orientation estimate obtained from the accelerometer data

Preferably a magnetometer step is performed by a gradient descent method following the step in the direction of the gradient obtained from the sequence of magnetometer data, which is analytically derived following the rules of the quaternion algebra, and especially normalised to minimise the difference between the true orientation and the assumed orientation estimate obtained from the magnetometer data.

Such a methodology does lead to especially fast and reliable convergence regarding assumed orientation.

The magnetometer and accelerometer steps may be combined into one step by mixing accelerometer and magnetometer gradients, to a field quaternion. Preferably, the sequence of gyroscope data, the sequence of accelerometer data and the sequence of magnetometer data is provided wirelessly, especially via Wi- Fi, especially from a glove or from a glove attachment. Such data is only provided if movement of the glove or the glove attachment and/or sensors is greater than a predetermined movement threshold and/or angular rate quaternion length is above a predetermined transmission threshold, which may be the same in step iii or different, and/or an accelerometer/magnetometer/gyroscope change in value of two consecutive samples is greater than some predetermined threshold, which may be the equivalent to the threshold in step ii or different therefrom. This enables reduction of power consumption while preserving most of the accuracy of the orientation estimation. The method of the invention is accurate enough to use quite a low data or sample rate and therefore such data can easily be provided wirelessly, even from a multitude of sensor triples. Data transmission rates may be in the range so that one triple value for each measurement, especially of the triple, is transmitted in the timespan of 10 to 100 ms. Sample rates of the sensors may differ and there may as well be different data transmit rates for different sensors.

Advantageously, repeatedly an updated assumed orientation quaternion is determined for a multitude of sensor triple data or sensor triples.

Assumed orientation quaternion, especially multiple assumed orientation quaternions each derived from one IMU MARG sensor set up can be used for further movement analysis. Such assumed orientation quaternions may be calculated simultaneously and/or in parallel. E.g., such IMU MARG sensors may be attached to body parts to detect their movement, e.g., the IMU MARG sensors may be attached to a person's finger, palm, arm, wrist, leg, and/or foot to detect movement thereof. Such assumed orientation quaternions related to body or body part movement may be used to recognize gestures, e.g., by use of neuronal networks. This can be used to reduce latency, especially below zero as gestures may even be recognized prior to being completed. Further, preferable ways to carry out the invention as well as exemplary results are discussed by reference to the following figures:

Fig. 1 shows a flow chart of the present inventive method;

Figs. 2.1 - 2.3 show example results of the method according to Fig. 1 in comparison to Madgwick and Wilson with:

Fig. 2.1 (a) to Fig. 2.1 (i) showing slow movement: roll (upper most), pitch (middle), and yaw (lowest) angle;

Fig. 2.2(a) to Fig. 2.2(i) showing medium movement: roll (upper most), pitch (middle), and yaw (lowest) angle;

Fig. 2.3(a) to Fig. 2.3(i) showing fast movement: roll (upper most), pitch (middle), and yaw (lowest) angle;

Fig. 3 is a more mathematical representation to perform steps i to vi, similar to Fig. 1 ;

Fig. 4 is a schematic view of a human wearing gloves with sensor arrays on its hands; and

Fig. 5 is a flow chart of a way to perform the method.

Referring now to Fig. 1 , there is shown a flow chart of a possible way to implement the method of the present invention.

The flow chart features reading of gyro data and, by use of a buffer and a threshold, deciding whether an IMU is stationary ("yes") or is moving or has moved ("no). In case of movement, the method branches to case A and where there is no movement to case B.

In branch A, accelerometer data is discarded and only magnetometer data is used to perform a single gradient descent. Magnetometer data is flattened by using a moving average filter and the cost function for the last step is calculated. Based on this, the weighting factor, here denoted as betam, is adapted and based on the weighting factor and the gradient, a quaternion is calculated. The quaternion is mixed with the calculated gyroscope quaternion, the integration of the quaternion derivative of the angular rate, to receive the output IMU orientation, the assumed orientation quaternion.

In branch B, the gyroscope data (the integration of the quaternion derivative of the angular rate) is assumed to be zero as no movement was detected. Magnetometer and accelerometer data are flattened by using moving average filters and the cost functions for the last step each are calculated. Based on these, the weighting factors, here denoted as betam and betaa, are adapted and based on the weighting factors and the gradients, a field quaternion is calculated. The field is mixed with the assumed orientation to receive the output IMU orientation, the (updated) assumed orientation quaternion.

Preferably the calculations and/or method described in this document is performed by or using at least one processor, especially a CPU and/or numerical processing unit and/or coprocessor. For such a computing unit incorporating the CPU and/or numerical processing unit and/or coprocessor may be used. The computing unit may include a memory to temporarily store non-volatile parts of or a full sequence of accelerometer, magnetometer and/or gyroscope data and/or assumed orientation quaternion, and/or all results calculated.

Figs. 2 show several results for different movement data and different parameters beta in the Madgwick algorithm. It can be seen that with the inventive method better orientation data can be derived, especially irrespectively of the form or intensity of movement.

Depicted are in the lines starting at the top:

Fig. 2.1 slow movement: roll (upper most), pitch (middle), and yaw (lowest) angle;

Fig. 2.2 medium movement: roll (upper most), pitch (middle), and yaw (lowest) angle; Fig. 2.3 fast movement: roll (upper most), pitch (middle), and yaw (lowest) angle, each from left to right with beta values for Madgwick and Wilson of (left to right) 0.01 , 1 .0, 10.0. As high value 2.0 was used. It is to be seen that the invention allows for fast convergence and reduces overshooting.

Fig. 3 provides a more mathematical representation of a way to perform steps i to vi of the present method similar to Fig. 1 , the following (additional) notation is used, while k is the rate time index: ῶ k is the raw gyroscope sample (before filtering) ᾶ k is the raw accelerometer sample (before filtering) the raw magnetometer sample (before filtering) ω k is the gyroscope sample after filtering and at the current iteration k a k is the accelerometer sample after filtering and at the current iteration k m k is the magnetometer sample after filtering and at the current iteration k m' k is the equivalent magnetometer measurement calculated according to the Eq. (6) from Wilson and at the current iteration k and is the direction of the gradient descent calculated using the Quaternion Derivatives [Mandic] for accelerometer and for mag- netometer, respectively w a,k and w m,k are regulated step sizes for QGD for acc and mag, respec- tively q w,k is the gyro quaternion is the field quaternion T s is the sampling period q est is the estimated orientation quaternion

Delay is the one sample delay

Eq. (6) from Wilson applied here means m'k = | m k x a k | . In the binary block taking the value of "IMU still?" 0 means false (the sensor is not still) and 1 means true (the sensor is still).

The delay saves the estimated orientation quaternion (assumed orientation quaternion, q est,k-1 ) until a new/updated one is calculated. depicts mixing may calculated by be

Nabla operator means in general the multidimensional gradient, in this case in the space, while * denotes the conjugate.

Fig. 4 shows a human wearing gloves with sensor arrays of accelerometer, magnetometer and gyroscope 1 .

Fig. 5 shows a flow chart of attaching sensors, e.g. at least an accelerometer, magnetometer and gyroscope to a human body, especially a limp or finger, e.g., by putting on a glove having such sensors, measuring data, especially accelerometer, magnetometer and gyroscope data and estimate the orientation of the human body or its parts, e.g., limp or finger and repeating such steps of acquiring data and estimate position at least once, preferably continuously, at least for some time. But data or sequences of data might be acquired prior to estimation as well if live estimation is not required. In such case estimation might as well be performed iteratively going through the sequences.