Physica A 569 (2021) 125770
Contents lists available at ScienceDirect
Physica A
journal homepage: www.elsevier.com/locate/physa
Quantifying the randomness of the forex market
Alfonso Delgado-Bonal
a,1,
∗
, Álvaro García López
b,1
a
Gentleman Scientist, 53C Crescent Road, Greenbelt, MD, USA
b
Universidad Rey Juan Carlos, Nonlinear Dynamics, Chaos and Complex System Group, Madrid, Spain
a r t i c l e i n f o
Article history:
Received 18 October 2020
Received in revised form 8 January 2021
Available online 18 January 2021
Keywords:
Complexity
Econophysics
Information Theory
Randomness
a b s t r a c t
Currency markets are international networks of participants opened all day during
weekdays without a supervisory entity. The precise value of an exchange pair is
determined by the decisions of the central banks and the behavior of the speculators,
whose actions can be determined on the spot or be related to previous decisions.
All those decisions affect the complexity and predictability of the system, which are
quantitatively analyzed in this paper. For this purpose, we compare the randomness
of the most traded currencies in the forex market using the Pincus Index. We extend
the development of this methodology to include multidimensionality in the embedding
dimension, to capture the influence of the past in current decisions and to analyze
different frequencies within the data with a multiscale approach. We show that, in
general, the forex market is more predictable using one hour ticks than using daily data
for the six major pairs, and present evidence suggesting that the variance is easier to
predict for longer time frames.
© 2021 Elsevier B.V. All rights reserved.
1. Introduction
The foreign exchange market (forex) is the largest market in the world and determines the exchange rates for every
currency. The participants in the market set the relative value of each currency pair by buying or selling positions, and
these actions are influenced by personal beliefs, trends or public announcements of central banks.
Unlike stock markets, forex is not being subject to an specific supervisory entity and is globally decentralized, open
to banks, commercial companies and private agents. The price of a currency pair at a given time is supposed to be a
reflection of economic factors, political conditions and the psychology of the participants.
The six most traded forex pairs analyzed in this paper are: EUR/USD (euro/US dollar); USD/JPY (US dollar/Japanese
yen); GBP/USD (British pound sterling/US dollar); AUD/USD (Australian dollar/US dollar); USD/CAD (US dollar/Canadian
dollar); USD/CNY (US dollar/Chinese renminbi).
The forex market has been the subject of intensive research for a long time, frequently focusing on the predictability
of its values but also on its variance. In this paper, we analyze the number of patterns within the data for the returns and
its variance in different timeframes, 1-hour (H1), 4-hours (H4) and daily.
Research on complexity includes a variety of algorithms and analysis techniques that usually come from the physical
or mathematical realm [1]. Complex systems are entangled by nonlinearly interacting elements and are found in different
fields such as the brain or financial markets. A review on the meaning of complexity and detailed analyses in those and
other processes can be found in [2].
∗
Corresponding author.
E-mail address: contact@adelgadobonal.com (A. Delgado-Bonal).
1
Both authors contributed equally to this work.
https://doi.org/10.1016/j.physa.2021.125770
0378-4371/© 2021 Elsevier B.V. All rights reserved.
A. Delgado-Bonal and Á.G. López Physica A 569 (2021) 125770
When dealing with the currency market, its movements can be separated into high-frequency variations [3] and slower
movements responsible for the trends [2]. This paper deals with the later, and explores its complexity relying on the
concept of entropy as defined in Information Theory and Kolmogorov complexity.
In the field of Information Theory, entropy is a magnitude that quantifies the uncertainty of a measure. On the
other hand, this paper follows the approach of Chaitin [4] and Kolmogorov [5] by defining complexity as in algorithmic
information theory, which takes into account the order of the points in a sequence. In this view, a chain is random if its
Kolmogorov complexity is at least equal to the length of the chain.
The benefit of the connection between information content and randomness is that it provides a way to quantify the
complexity of a dataset without relying on models or hypothesis about the process generating the data. By comparing the
entropy of our system with the maximum entropy rate possible, we can determine the degree of randomness of a series;
a complex (total random) process is defined as that process lacking pattern repetition.
The use of the entropy rate to study the complexity of a time series is not limited to stochastic processes. Sinai [6]
introduced the concept of entropy to describe the structural similarity between different dynamic systems that preserve
the measurements, giving a generalization of Shannon entropy for dynamic systems, known as Kolmogorov–Sinai entropy
(KS). Unfortunately, KS entropy is sometimes undefined for limited and noisy measurements of a signal represented in a
data series.
To overcome that limitation, Grassberger and Procaccia [7] used the Renyi entropy to define the correlation integral,
which in turn was used by Eckmann and Ruelle [8] to define the φ functions as a conditional probability. This ER entropy
is an exact estimation of the entropy of the system. Building upon those φ functions, Pincus [9] described the methodology
of ApEn, useful for limited and noisy data, providing a hierarchy of randomness based on the different patterns and their
repetitions.
ApEn measures the logarithmic probability that nearby pattern runs remain close in the next incremental comparison:
low ApEn values reflect that the system is very persistent, repetitive and predictive, with apparent patterns that repeat
themselves throughout of the series, while high values means complexity in the sense of independence between the data
and a low number of repeated patterns. The readers are encouraged to read a recent comprehensive tutorial on these
algorithms [10].
To use Approximate Entropy, it is necessary to specify two parameters, the embedding dimension (m) and the tolerance
of the measure (r), determined as a percentage of the standard deviation. Once the calculations have been performed, the
result of the algorithm is a positive real number, with higher values indicating more randomness. However, those values
are dependent on the characteristics of the dataset such as the influence of the past in the future prices or the volatility
of the prices.
In order to obtain a measure of randomness suitable for comparisons between evolving datasets, the Pincus Index (PI)
was introduced as a measure of the distance between a dataset and the maximum possible randomness of that system [11].
A value of PI equal to zero implies a totally ordered and completely predictable system, whereas a value equal to or greater
than one implies total randomness and unpredictability. The added benefit of the Pincus Index is that, unlike ApEn, it is
suitable for comparisons between different markets. This paper completes the development of that index by introducing
different kinds of multidimensionality in the measure. Thus, knowledge of the PI would be useful to fully understand the
concepts here presented and how the several levels of complexity of this measure are captured.
The Pincus Index was designed [11] to be independent on the parameter r by choosing the maximum value of
Approximate Entropy (MaxApEn), but the index is still dependent on the selection of the embedding dimension (m). This
parameter is related to the memory of the system and accounts for the length of the patterns compared in the sequence.
Techniques to determine the optimum value of the embedding dimension include the use the mutual information and false
nearest neighbor method [12–14], but since different markets may have different embedding dimensions, the comparisons
with a fixed m could be biased. To account for that possibility, we follow Bolea and coauthors [15] in the definition of
a Multidimensional index. Since such an index was based on MaxApEn, its extrapolation to a Multidimensional Pincus
Index is straightforward and provides a parameter-free index which allows for comparisons between evolving systems.
Besides multidimensionality in embedding dimension, dynamic systems may be composed of processes at different
frequencies with correlations at multiple time scales. Therefore, in the characterization of complexity, the comparison
of different frequencies may lead to incorrect conclusions. Costa and coauthors [16] proposed a multiscale procedure
to capture those correlations, showing its efficiency in distinguishing complexities in different dynamical regimes. To
describe the complexity of a time series at different levels, Costa and coauthors [17] generalized the multiscale procedure
to consider the complexity of higher statistical moments of time series. Here, we extend that methodology to create a new
Multiscale Pincus Index, showing how it is useful to correctly quantify the complexity of trading in different timeframes
and different statistical moments.
2. Methods and results
2.1. On the calculation of the Pincus Index
The Pincus Index (PI) captures the distance from a situation of total randomness for a given dataset, measured against
shuffled versions of the same data. To better quantify complexity and provide an index that is independent of the tolerance
2
A. Delgado-Bonal and Á.G. López Physica A 569 (2021) 125770
Fig. 1. ApEn, SampEn, and asymptotic lines depending on the alphabet for 50 pseudo-random binary (left) and decimal (right) sequences.
r, it is constructed based on the maximum value of Approximate Entropy (MaxApEn). The steps to compute the PI include
the determination of the MaxApEn of the original sequence and the MaxApEn of bootstrapped versions. Then, we use the
median value (50% percentile) of the empirical distribution of the bootstrapped versions to calculate the value of the
Pincus Index, and the 5% and 95% percentiles of the empirical cumulative distribution function to calculate the extremes
of the index. The rationale is simple: if the degree of randomness of the original sequence is similar to the shuffled
versions, the PI will be close to one, indicating randomness. If, on the other hand, the original sequence is ordered, the
PI will capture the distance from randomness as a fraction. For a detailed explanation of the methodology and several
examples of application, the reader is encouraged to see [10,11].
The Pincus Index is based on Approximate Entropy. When the number of data (N) is large, ApEn can be approximated
by Eq. (1). The error committed in this approximation is estimated to be smaller than 0.05 for N −m+1 > 90 and smaller
than 0.02 for N − m + 1 > 283 [18].
ApEn(m, r, N) ≃
−
1
N − m
N−m
i=1
log
N−m
j=1
[times that d[|x
m+1
(j) − x
m+1
(i)|] < r]
N−m
j=1
[times that d[|x
m
(j) − x
m
(i)|] < r]
.
(1)
where m is the length of the vectors being compared, and d measures the scalar distance between the vectors in a
component-wise way.
The Sample Entropy (SampEn) algorithm has been designed to avoid the self-bias included in ApEn [18], which is
mathematically formulated as [10]:
SampEn(m, r, N) =
− log
N−m
i=1
N−m
j=1,j=i
[times that d[|x
m+1
(j) − x
m+1
(i)|] < r]
N−m
i=1
N−m
j=1,j=i
[times that d[|x
m
(j) − x
m
(i)|] < r]
(2)
It is often said that SampEn is largely independent on the number of points because, unlike ApEn, it does not include
a prefactor
1
N−m
. However, it must be noticed that such independence is only true for homogeneous series, and it does
not hold for general situations [19]. In general, randomness depends for both algorithms on m and N [20].
In Fig. 1 we show the different behavior of ApEn and SampEn for 50 pseudo-random binary (left) and decimal (right)
chains using m = 2; we use r < 1 to make the analysis independent of this parameter, given their well-defined alphabet.
As it can be seen, the mean value of SampEn reaches the asymptotic limit of log k faster but with a larger standard
deviation than ApEn.
In the construction of the Pincus Index, we calculate the ratio MaxApEn
original
/MaxApEn
shuffled
. Since those quantities
are calculated using the same values of m and N, the ratio between them does not include the prefactor
1
N−m
appearing
in ApEn in Eq. (1). This fact makes the PI independent of the number of points in the same way that SampEn (i.e., for
white noise, or a homogeneously generated sequence, it captures the randomness independently of N).
Another reason for the construction of SampEn was the self-counting introduced in the calculation of ApEn: note that
the definition of SampEn explicitly avoids that situation by limiting j = i in Eq. (2). That bias can be as high as 20% or 30%
if the number of points is low [18]. In this regard, since the PI is constructed as a ratio and the bias in ApEn is present in
both the nominator and the denominator, the overall bias is modulated and severely corrected, providing a better measure
of complexity. It should be emphasized that the PI does not measure randomness specifically but how far away a series
is from total randomness.
3
A. Delgado-Bonal and Á.G. López Physica A 569 (2021) 125770
Fig. 2. Value of r for which MaxApEn is reached for the original series (red line) and the average of one hundred shuffled versions (black line). The
average of the standard deviations (bars) for the shuffled versions is 0.038 and increases with the embedding dimension.
2.2. The threshold: r
max
and MaxApEn
The use of an incorrect parameter selection when using ApEn or SampEn can lead to inaccurate estimations of the
complexity of datasets. By means of MaxApEn we can prevent the arbitrary selection of the threshold of r, which changes
depending on the complexity of the sequence.
Restrepo et al. [21] showed that the combined use of MaxApEn and r
max
can help to correctly characterize the
complexity. Using a dataset containing daily values of EURUSD from 2006 to 2010, we show in Fig. 2 that r
max
changes with
the embedding dimension selected. The distance between the maximum value of the threshold for the original (red line)
and the shuffled series (black line) shows that using a fixed common valued for the threshold would lead to misleading
results. Thus, even though the recommended range for r is commonly [0.1σ , 0.25σ ], that region does not guarantee to
capture the complexity correctly for all the values of the embedding dimension. It is advised to use a value equal to or
greater than the value of r
max
[10] to assure the relative consistency; the comparison with different values of r beyond
the maximum would lead to the same qualitative characterization of the order of the system.
As explained in [21], the differences in r
max
for the original and shuffled versions can be used as a mean to discern
between systems in noisy datasets with low number of samples N. Albeit in this work we focus on the development of
the Pincus Index, we take the opportunity to recall that, for some dynamical regimes, these combined techniques could
provide a better characterization of the systems, and show that the recommended range may not be adequate depending
on the embedding dimension m.
2.3. The embedding dimension: multidimensional analysis
In the methodology to calculate the Pincus Index, the tolerance r is automatically selected as the value which
maximizes Approximate Entropy [11]. However, the selection of the embedding dimension is a requirement for the
calculations. The embedding dimension determines the length of the patterns being compared, and it is related to how
much information from the past is used to determine the future values. In the search of a parameter-free application
of Approximate Entropy, Bolea et al. [15] proposed the use of MaxApEn combined with a multidimensional analysis by
adding the contribution of MaxApEn over a wide range of embedding dimensions to capture the influence of previous
values.
Since a priori the memory of the system is unknown and it may change in evolving datasets like the forex markets,
adopting the same methodology as Bolea and coauthors, it is straight forward to build a Multidimensional Pincus Index
(MPI) independent of both r and m, by defining:
MPI =
m
max
m
i
=1
MaxApEn
original
(m
i
)
m
max
m=1
MaxApEn
shuffled
(m
i
)
(3)
We illustrate the behavior of this new multidimensional index in Fig. 3 using the EURUSD exchange rate as an example.
Fig. 3 (left) shows the MaxApEn for different embedding dimensions for the original series (black line) and pseudo-
randomized versions of the same data (red) for a dataset containing daily values of EURUSD from 2006 to 2010. Based
on those values, we show how the MPI changes when we consider only the previous m values using Eq. (3). We observe
that, by adding the contribution of larger embedding dimensions, the MPI varies to capture the increased information
in the complexity of the series. The right side of Fig. 3 shows the MPI accounting for the contribution of the embedding
dimension up to 15 (MPI(m
max
= 15)) for rolling windows of four years of EURUSD daily exchange rate, i.e., approximately
N ∼ 1000 points.
4
A. Delgado-Bonal and Á.G. López Physica A 569 (2021) 125770
Fig. 3. Multidimensional Pincus Index.
The rationale for the inclusion of multidimensionality is its ability to capture complexity in a greater extent, as shown
by Bolea et al. [15]. Specifically for the forex or stock markets, or when drawing comparisons between different systems, it
is not guaranteed that the optimal value of the embedding dimension would be the same. In general, randomness depends
on m and N, as shown by Pincus and coauthors when they defined the maximum {m, N}-randomness [20,22]. The value
of the Pincus Index is its aptness to make comparisons between systems by measuring the distance of each series against
the maximum randomness of each alphabet. We shall remember that both ApEn and SampEn provide relative values, and
may be unsuitable for comparisons. By including multidimensionality in the definition of the MPI, we obtain an index
independent of preselected parameter values for both r and m which can be used with evolving datasets.
2.4. Sampling frequency: multiscale entropy
Another variable must be taken into account in order to capture complexity in all of its forms, which is the different
frequencies within the data. It is not uncommon for dynamical systems to be composed of subprocesses emerging at
different time scales. That situation is often observed in the markets when the trend at a certain frequency domain, let
us say 15 min, is not the same (or even the complete opposite) as the trend at 1 day data.
To account for that possibility, Costa et al. [16] designed the multiscale entropy (MSE) procedure based on the approach
proposed by Zang [23,24]. This measure is based on a weighted sum of scale dependent entropies, and it has been used
extensively since its appearance in the literature [25]. The main idea is the construction of coarse-grained time series
determined by a certain scale factor τ , averaging different time scales from the original time series. The coarse-graining
procedure reduces the length of the sequence by a scale factor τ , obtaining a coarse-grained time of length N/τ , with
N the original length. Thus, the larger the scale factor used, the shorter the resulting length of the coarse-grained time
series.
This procedure has become a prevailing method to quantify the complexity of data series and it has been vastly applied
in many different research fields, including finances [26]. After the creation of the coarse-grained sequences, the entropy
of each sequence is calculated and added up to obtain a multiscale entropy value. More detailed instructions of the
methodology can be found in Costa and coauthors works [16,27].
The MSE methodology has generally been applied in conjunction to Sample Entropy, given the above-mentioned fact
that is less dependent on the time series length since it does not include a prefactor in Eq. (2). However, similarly to
the comparison of different time series, different time scales may have different alphabets and the comparisons using
the same parameters may be biased. Some traded time frames will show higher variability, while the variations at
different frequencies may show lower changes and averaged values. Furthermore, Sample Entropy uses a fixed value
of the tolerance filter r which may not be adequate for all frequencies. This hinders the applicability of Sample Entropy
to characterize the randomness level appropriately.
As seen in the previous sections, the value of r which captures the maximum complexity is different for each sequence;
since the Pincus Index is based on MaxApEn, which automatically adapts to the maximum complexity of each frequency,
this index is able to capture the distance from total randomness of the different frequencies. As an example, Fig. 4 shows
the results of the Pincus Index for values traded Daily, at four hours (H4) and at one hour (H1) frequencies for m = 2.
We present the results for the six major traded pairs to display the evolution of the different frequencies.
In a previous communication we showed the effect of including more or less number of points (see Supplementary
Information in [11]). In this paper, our interest lies on characterizing the different frequencies knowing that approximately
the same number of points are used to draw comparisons about their complexity. To that end, we use rolling windows of
approximately N ∼ 1000 for all frequencies showed in Fig. 4, corresponding to four years in daily values, eight months in
5