sam/pyFTS
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity
pyFTS/pyFTS/notebooks/Severiano et al - HighOrderFTS.ipynb
Petrônio Cândido a9a07204dc Configuring the new tag for pyPI release
2018-03-06 19:47:14 -03:00

357 KiB
Raw Blame History

High Order Fuzzy Time Series

Severiano, S. A. Jr; Silva, P. C. L.; Sadaei, H. J.; Guimarães, F. G. Very Short-term Solar Forecasting using Fuzzy Time Series. 2017 IEEE International Conference on Fuzzy Systems. DOI10.1109/FUZZ-IEEE.2017.8015732

Common Imports

In [1]:
import matplotlib.pylab as plt
from pyFTS.benchmarks import benchmarks as bchmk
from pyFTS.models import hofts

from pyFTS.common import Transformations
tdiff = Transformations.Differential(1)

%pylab inline

/usr/local/lib/python3.6/dist-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
  from pandas.core import datetools

Populating the interactive namespace from numpy and matplotlib

/usr/lib/python3/dist-packages/IPython/core/magics/pylab.py:161: UserWarning: pylab import has clobbered these variables: ['plt']
`%matplotlib` prevents importing * from pylab and numpy
  "\n`%matplotlib` prevents importing * from pylab and numpy"

Data Loading

In [2]:
from pyFTS.data import Enrollments

enrollments = Enrollments.get_data()

---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
<ipython-input-2-c717b3ec41cd> in <module>()
      1 from pyFTS.data import Enrollments
      2 
----> 3 enrollments = Enrollments.get_data()
      4 

/usr/local/lib/python3.6/dist-packages/pyFTS/data/Enrollments.py in get_data()
      5 
      6 def get_data():
----> 7     dat = get_dataframe()
      8     dat = np.array(dat["Enrollments"])
      9     return dat

/usr/local/lib/python3.6/dist-packages/pyFTS/data/Enrollments.py in get_dataframe()
     13     dat = common.get_dataframe('data/Enrollments.csv',
     14                                'https://github.com/petroniocandido/pyFTS/raw/8f20f3634aa6a8f58083bdcd1bbf93795e6ed767/pyFTS/data/Enrollments.csv',
---> 15                                sep=";")
     16     return dat

/usr/local/lib/python3.6/dist-packages/pyFTS/data/common.py in get_dataframe(path, url, sep, compression)
     14         return pd.read_csv(filename, sep=sep, compression=compression)
     15     else:
---> 16         request.urlretrieve(url, filename)
     17         return pd.read_csv(filename, sep=sep, compression=compression)
     18 

/usr/lib/python3.6/urllib/request.py in urlretrieve(url, filename, reporthook, data)
    256         # Handle temporary file setup.
    257         if filename:
--> 258             tfp = open(filename, 'wb')
    259         else:
    260             tfp = tempfile.NamedTemporaryFile(delete=False)

PermissionError: [Errno 13] Permission denied: '/usr/local/lib/python3.6/dist-packages/pyFTS/data/Enrollments.csv'

Exploring the partitioning effects on original data

In [6]:
tmp = bchmk.simpleSearch_RMSE(enrollments, enrollments, hofts.HighOrderFTS, range(1,20), [1, 2, 3], tam=[10, 5])

Exploring the partitioning effects on transformed data

In [7]:
tmp = bchmk.simpleSearch_RMSE(enrollments, enrollments, hofts.HighOrderFTS, range(1,20), [1, 2, 3], 
                              transformation=tdiff, tam=[10, 5])

Comparing the partitioning schemas

In [8]:
from pyFTS.partitioners import Grid, Util as pUtil

fuzzy_sets = Grid.GridPartitioner(data=enrollments, npart=12)
fuzzy_sets2 = Grid.GridPartitioner(data=enrollments, npart=5, transformation=tdiff)

pUtil.plot_partitioners(enrollments, [fuzzy_sets,fuzzy_sets2])

Fitting a model on original data

In [9]:
model1 = hofts.HighOrderFTS("FTS", partitioner=fuzzy_sets)
model1.fit(enrollments, order=3)

print(model1)

[ 17:43:07] Start training
[ 17:43:07] Finish training
High Order FTS:
A1,A2,A2 -> A3,A4
A1,A2,A3 -> A3,A4
A1,A3,A2 -> A3,A4
A1,A3,A3 -> A3,A4
A10,A10,A10 -> A8,A9
A10,A10,A9 -> A8,A9
A10,A9,A10 -> A8,A9
A10,A9,A9 -> A8,A9
A2,A2,A2 -> A3,A4
A2,A2,A3 -> A3,A4,A5
A2,A2,A4 -> A4,A5
A2,A3,A2 -> A3,A4
A2,A3,A3 -> A3,A4,A5
A2,A3,A4 -> A4,A5
A2,A3,A5 -> A4,A5
A2,A4,A4 -> A4,A5
A2,A4,A5 -> A4,A5
A3,A2,A3 -> A4,A5
A3,A2,A4 -> A4,A5
A3,A3,A3 -> A4,A5
A3,A3,A4 -> A4,A5
A3,A3,A5 -> A4,A5
A3,A4,A4 -> A4,A5
A3,A4,A5 -> A4,A5
A3,A5,A4 -> A4,A5
A3,A5,A5 -> A4,A5
A4,A4,A4 -> A4,A5,A6
A4,A4,A5 -> A4,A5,A6,A7
A4,A4,A6 -> A6,A7
A4,A5,A4 -> A4,A5,A6
A4,A5,A5 -> A4,A5,A6,A7
A4,A5,A6 -> A6,A7,A8,A9
A4,A5,A7 -> A6,A7,A8,A9
A4,A6,A6 -> A6,A7,A8,A9
A4,A6,A7 -> A6,A7,A8,A9
A5,A4,A4 -> A4,A5,A6
A5,A4,A5 -> A4,A5,A6,A7
A5,A4,A6 -> A6,A7
A5,A5,A4 -> A4,A5,A6
A5,A5,A5 -> A4,A5,A6,A7
A5,A5,A6 -> A6,A7,A8,A9
A5,A5,A7 -> A6,A7,A8,A9
A5,A6,A6 -> A5,A6,A7,A8,A9
A5,A6,A7 -> A5,A6,A7,A8,A9
A5,A6,A8 -> A10,A9
A5,A6,A9 -> A10,A9
A5,A7,A6 -> A5,A6
A5,A7,A7 -> A5,A6
A5,A7,A8 -> A10,A9
A5,A7,A9 -> A10,A9
A6,A4,A4 -> A4,A5
A6,A4,A5 -> A4,A5
A6,A5,A4 -> A4,A5
A6,A5,A5 -> A4,A5
A6,A6,A4 -> A4,A5
A6,A6,A5 -> A4,A5
A6,A6,A6 -> A4,A5,A6
A6,A6,A7 -> A5,A6
A6,A6,A8 -> A10,A9
A6,A6,A9 -> A10,A9
A6,A7,A5 -> A4,A5
A6,A7,A6 -> A4,A5,A6
A6,A7,A7 -> A5,A6
A6,A7,A8 -> A10,A9
A6,A7,A9 -> A10,A9
A6,A8,A10 -> A10,A9
A6,A8,A9 -> A10,A9
A6,A9,A10 -> A10,A9
A6,A9,A9 -> A10,A9
A7,A5,A4 -> A4,A5
A7,A5,A5 -> A4,A5
A7,A6,A4 -> A4,A5
A7,A6,A5 -> A4,A5
A7,A6,A6 -> A4,A5
A7,A7,A5 -> A4,A5
A7,A7,A6 -> A4,A5
A7,A8,A10 -> A10,A9
A7,A8,A9 -> A10,A9
A7,A9,A10 -> A10,A9
A7,A9,A9 -> A10,A9
A8,A10,A10 -> A10,A9
A8,A10,A9 -> A10,A9
A8,A9,A10 -> A10,A9
A8,A9,A9 -> A10,A9
A9,A10,A10 -> A10,A8,A9
A9,A10,A9 -> A10,A8,A9
A9,A9,A10 -> A10,A8,A9
A9,A9,A9 -> A10,A8,A9

Fitting a model on transformed data

In [10]:
model2 = hofts.HighOrderFTS("FTS Diff", partitioner=fuzzy_sets2)
model2.append_transformation(tdiff)
model2.fit(enrollments, order=3)

print(model2)

[ 17:43:16] Start training
[ 17:43:16] Finish training
High Order FTS:
A0,A1,A2 -> A2,A3
A0,A1,A3 -> A2,A3
A0,A2,A2 -> A2,A3
A0,A2,A3 -> A2,A3,A4
A0,A2,A4 -> A4
A0,A3,A3 -> A4
A0,A3,A4 -> A4
A1,A0,A1 -> A2,A3
A1,A0,A2 -> A2,A3
A1,A1,A0 -> A1,A2
A1,A1,A1 -> A1,A2,A3
A1,A1,A2 -> A2,A3
A1,A1,A3 -> A2,A3
A1,A1,A4 -> A2,A3
A1,A2,A0 -> A1,A2
A1,A2,A1 -> A1,A2
A1,A2,A2 -> A1,A2,A3,A4
A1,A2,A3 -> A2,A3,A4
A1,A2,A4 -> A2,A3,A4
A1,A3,A1 -> A1,A2
A1,A3,A2 -> A1,A2,A3,A4
A1,A3,A3 -> A2,A3,A4
A1,A3,A4 -> A4
A1,A4,A2 -> A2,A3
A1,A4,A3 -> A2,A3
A2,A0,A1 -> A2,A3
A2,A0,A2 -> A2,A3,A4
A2,A0,A3 -> A3,A4
A2,A1,A0 -> A1,A2
A2,A1,A1 -> A0,A1,A2,A3,A4
A2,A1,A2 -> A0,A1,A2,A3,A4
A2,A1,A3 -> A1,A2,A3,A4
A2,A1,A4 -> A2,A3
A2,A2,A0 -> A1,A2,A3
A2,A2,A1 -> A0,A1,A2,A3,A4
A2,A2,A2 -> A0,A1,A2,A3,A4
A2,A2,A3 -> A1,A2,A3,A4
A2,A2,A4 -> A2,A3
A2,A3,A0 -> A2,A3
A2,A3,A1 -> A1,A2,A3
A2,A3,A2 -> A1,A2,A3,A4
A2,A3,A3 -> A1,A2,A3,A4
A2,A3,A4 -> A2,A3
A2,A4,A2 -> A1,A2,A3
A2,A4,A3 -> A1,A2,A3
A2,A4,A4 -> A2,A3
A3,A0,A2 -> A3,A4
A3,A0,A3 -> A3,A4
A3,A1,A1 -> A0,A1,A3,A4
A3,A1,A2 -> A0,A1,A2,A3,A4
A3,A1,A3 -> A1,A2,A3,A4
A3,A2,A0 -> A2,A3
A3,A2,A1 -> A0,A1,A2,A3,A4
A3,A2,A2 -> A0,A1,A2,A3,A4
A3,A2,A3 -> A0,A1,A2
A3,A3,A0 -> A2,A3
A3,A3,A1 -> A1,A2,A3
A3,A3,A2 -> A0,A1,A2
A3,A3,A3 -> A0,A1,A2
A3,A3,A4 -> A2,A3
A3,A4,A2 -> A1,A2,A3,A4
A3,A4,A3 -> A1,A2,A3,A4
A3,A4,A4 -> A2,A3
A4,A2,A1 -> A1,A2
A4,A2,A2 -> A0,A1,A2
A4,A2,A3 -> A0,A1,A2,A3
A4,A2,A4 -> A2,A3
A4,A3,A1 -> A1,A2
A4,A3,A2 -> A0,A1,A2
A4,A3,A3 -> A0,A1,A2,A3
A4,A3,A4 -> A2,A3
A4,A4,A2 -> A3,A4
A4,A4,A3 -> A3,A4

Using the models

In [11]:
model1.predict(enrollments)
Out[11]:
[14923.233333333337,
 15319.950000000004,
 15319.950000000004,
 16113.383333333339,
 16113.383333333339,
 17700.250000000007,
 17303.53333333334,
 16113.383333333339,
 15319.950000000004,
 15319.950000000004,
 15319.950000000004,
 16113.383333333339,
 16113.383333333339,
 17700.250000000007,
 17303.53333333334,
 19287.116666666676,
 19287.116666666676,
 18890.40000000001,
 18890.40000000001,
 18890.40000000001]
In [12]:
model2.predict(enrollments)
Out[12]:
[12941.02,
 14511.2,
 15028.14,
 15126.2,
 15171.14,
 15182.08,
 16375.14,
 16734.2,
 16944.38,
 15248.2,
 15806.32,
 14960.2,
 14978.2,
 15058.02,
 16427.14,
 17718.14,
 18538.14,
 19390.260000000002,
 19152.2,
 18691.2]

Comparing the models

In [13]:
bchmk.plot_compared_series(enrollments, [model1, model2], bchmk.colors, intervals=False)
In [14]:
bchmk.print_point_statistics(enrollments, [model1, model2])

Model		& Order     & RMSE		& SMAPE      & Theil's U		\\ 
HOFTSFTS		& 3		& 466.65		& 1.06		& 0.76	\\ 
HOFTSFTS Diff		& 3		& 1029.5		& 2.66		& 1.68	\\

Residual Analysis

In [ ]:
from pyFTS.benchmarks import ResidualAnalysis as ra

ra.plot_residuals(enrollments, [model1, model2])
In [ ]:
In [ ]: