sam/pyFTS
1
0
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity
pyFTS/pyFTS/notebooks/HighOrderFTS.ipynb
Petrônio Cândido 7a2dd7e54c Notebooks and method bugfixes and refactoring
2018-02-27 18:30:20 -03:00

383 KiB
Raw Blame History

High Order Fuzzy Time Series

Common Imports

In [1]:
import matplotlib.pylab as plt
from pyFTS.benchmarks import benchmarks as bchmk
from pyFTS.models import hofts

%pylab inline

Populating the interactive namespace from numpy and matplotlib

/usr/local/lib/python3.6/dist-packages/statsmodels/compat/pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
  from pandas.core import datetools
/usr/lib/python3/dist-packages/IPython/core/magics/pylab.py:161: UserWarning: pylab import has clobbered these variables: ['plt']
`%matplotlib` prevents importing * from pylab and numpy
  "\n`%matplotlib` prevents importing * from pylab and numpy"

Data Loading

In [2]:
from pyFTS.data import Enrollments

enrollments = Enrollments.get_data()

Exploring the partitioning effects on original data

In [3]:
tmp = bchmk.simpleSearch_RMSE(enrollments, enrollments, hofts.HighOrderFTS, range(1,20), [1, 2, 3], tam=[10, 5])

Exploring the partitioning effects on transformed data

In [4]:
from pyFTS.common import Transformations
diff = Transformations.Differential(1)

tmp = bchmk.simpleSearch_RMSE(enrollments, enrollments, hofts.HighOrderFTS, range(1,20), [1, 2, 3], 
                              transformation=diff, tam=[10, 5])

Comparing the partitioning schemas

In [5]:
from pyFTS.partitioners import Grid, Util as pUtil

fuzzy_sets = Grid.GridPartitioner(enrollments, 12)
fuzzy_sets2 = Grid.GridPartitioner(enrollments, 5, transformation=diff)

pUtil.plot_partitioners(enrollments, [fuzzy_sets,fuzzy_sets2])

Fitting a model on original data

In [13]:
model1 = hofts.HighOrderFTS("FTS", partitioner=fuzzy_sets)
model1.fit(enrollments, order=3)

print(model1)

High Order FTS:
A1, A2, A2 -> A3,A4
A1, A2, A3 -> A3,A4
A1, A3, A2 -> A3,A4
A1, A3, A3 -> A3,A4
A10, A10, A10 -> A8,A9
A10, A10, A9 -> A8,A9
A10, A9, A10 -> A8,A9
A10, A9, A9 -> A8,A9
A2, A2, A2 -> A3,A4
A2, A2, A3 -> A3,A4,A5
A2, A2, A4 -> A4,A5
A2, A3, A2 -> A3,A4
A2, A3, A3 -> A3,A4,A5
A2, A3, A4 -> A4,A5
A2, A3, A5 -> A4,A5
A2, A4, A4 -> A4,A5
A2, A4, A5 -> A4,A5
A3, A2, A3 -> A4,A5
A3, A2, A4 -> A4,A5
A3, A3, A3 -> A4,A5
A3, A3, A4 -> A4,A5
A3, A3, A5 -> A4,A5
A3, A4, A4 -> A4,A5
A3, A4, A5 -> A4,A5
A3, A5, A4 -> A4,A5
A3, A5, A5 -> A4,A5
A4, A4, A4 -> A4,A5,A6
A4, A4, A5 -> A4,A5,A6,A7
A4, A4, A6 -> A6,A7
A4, A5, A4 -> A4,A5,A6
A4, A5, A5 -> A4,A5,A6,A7
A4, A5, A6 -> A6,A7,A8,A9
A4, A5, A7 -> A6,A7,A8,A9
A4, A6, A6 -> A6,A7,A8,A9
A4, A6, A7 -> A6,A7,A8,A9
A5, A4, A4 -> A4,A5,A6
A5, A4, A5 -> A4,A5,A6,A7
A5, A4, A6 -> A6,A7
A5, A5, A4 -> A4,A5,A6
A5, A5, A5 -> A4,A5,A6,A7
A5, A5, A6 -> A6,A7,A8,A9
A5, A5, A7 -> A6,A7,A8,A9
A5, A6, A6 -> A5,A6,A7,A8,A9
A5, A6, A7 -> A5,A6,A7,A8,A9
A5, A6, A8 -> A10,A9
A5, A6, A9 -> A10,A9
A5, A7, A6 -> A5,A6
A5, A7, A7 -> A5,A6
A5, A7, A8 -> A10,A9
A5, A7, A9 -> A10,A9
A6, A4, A4 -> A4,A5
A6, A4, A5 -> A4,A5
A6, A5, A4 -> A4,A5
A6, A5, A5 -> A4,A5
A6, A6, A4 -> A4,A5
A6, A6, A5 -> A4,A5
A6, A6, A6 -> A4,A5,A6
A6, A6, A7 -> A5,A6
A6, A6, A8 -> A10,A9
A6, A6, A9 -> A10,A9
A6, A7, A5 -> A4,A5
A6, A7, A6 -> A4,A5,A6
A6, A7, A7 -> A5,A6
A6, A7, A8 -> A10,A9
A6, A7, A9 -> A10,A9
A6, A8, A10 -> A10,A9
A6, A8, A9 -> A10,A9
A6, A9, A10 -> A10,A9
A6, A9, A9 -> A10,A9
A7, A5, A4 -> A4,A5
A7, A5, A5 -> A4,A5
A7, A6, A4 -> A4,A5
A7, A6, A5 -> A4,A5
A7, A6, A6 -> A4,A5
A7, A7, A5 -> A4,A5
A7, A7, A6 -> A4,A5
A7, A8, A10 -> A10,A9
A7, A8, A9 -> A10,A9
A7, A9, A10 -> A10,A9
A7, A9, A9 -> A10,A9
A8, A10, A10 -> A10,A9
A8, A10, A9 -> A10,A9
A8, A9, A10 -> A10,A9
A8, A9, A9 -> A10,A9
A9, A10, A10 -> A10,A8,A9
A9, A10, A9 -> A10,A8,A9
A9, A9, A10 -> A10,A8,A9
A9, A9, A9 -> A10,A8,A9

Fitting a model on transformed data

In [14]:
model2 = hofts.HighOrderFTS("FTS Diff", partitioner=fuzzy_sets2)
model2.append_transformation(diff)
model2.fit(enrollments, order=3)

print(model2)

High Order FTS:
A0, A1, A2 -> A2,A3
A0, A1, A3 -> A2,A3
A0, A2, A2 -> A2,A3
A0, A2, A3 -> A2,A3,A4
A0, A2, A4 -> A4
A0, A3, A3 -> A4
A0, A3, A4 -> A4
A1, A0, A1 -> A2,A3
A1, A0, A2 -> A2,A3
A1, A1, A0 -> A1,A2
A1, A1, A1 -> A1,A2,A3
A1, A1, A2 -> A2,A3
A1, A1, A3 -> A2,A3
A1, A1, A4 -> A2,A3
A1, A2, A0 -> A1,A2
A1, A2, A1 -> A1,A2
A1, A2, A2 -> A1,A2,A3,A4
A1, A2, A3 -> A2,A3,A4
A1, A2, A4 -> A2,A3,A4
A1, A3, A1 -> A1,A2
A1, A3, A2 -> A1,A2,A3,A4
A1, A3, A3 -> A2,A3,A4
A1, A3, A4 -> A4
A1, A4, A2 -> A2,A3
A1, A4, A3 -> A2,A3
A2, A0, A1 -> A2,A3
A2, A0, A2 -> A2,A3,A4
A2, A0, A3 -> A3,A4
A2, A1, A0 -> A1,A2
A2, A1, A1 -> A0,A1,A2,A3,A4
A2, A1, A2 -> A0,A1,A2,A3,A4
A2, A1, A3 -> A1,A2,A3,A4
A2, A1, A4 -> A2,A3
A2, A2, A0 -> A1,A2,A3
A2, A2, A1 -> A0,A1,A2,A3,A4
A2, A2, A2 -> A0,A1,A2,A3,A4
A2, A2, A3 -> A1,A2,A3,A4
A2, A2, A4 -> A2,A3
A2, A3, A0 -> A2,A3
A2, A3, A1 -> A1,A2,A3
A2, A3, A2 -> A1,A2,A3,A4
A2, A3, A3 -> A1,A2,A3,A4
A2, A3, A4 -> A2,A3
A2, A4, A2 -> A1,A2,A3
A2, A4, A3 -> A1,A2,A3
A2, A4, A4 -> A2,A3
A3, A0, A2 -> A3,A4
A3, A0, A3 -> A3,A4
A3, A1, A1 -> A0,A1,A3,A4
A3, A1, A2 -> A0,A1,A2,A3,A4
A3, A1, A3 -> A1,A2,A3,A4
A3, A2, A0 -> A2,A3
A3, A2, A1 -> A0,A1,A2,A3,A4
A3, A2, A2 -> A0,A1,A2,A3,A4
A3, A2, A3 -> A0,A1,A2
A3, A3, A0 -> A2,A3
A3, A3, A1 -> A1,A2,A3
A3, A3, A2 -> A0,A1,A2
A3, A3, A3 -> A0,A1,A2
A3, A3, A4 -> A2,A3
A3, A4, A2 -> A1,A2,A3,A4
A3, A4, A3 -> A1,A2,A3,A4
A3, A4, A4 -> A2,A3
A4, A2, A1 -> A1,A2
A4, A2, A2 -> A0,A1,A2
A4, A2, A3 -> A0,A1,A2,A3
A4, A2, A4 -> A2,A3
A4, A3, A1 -> A1,A2
A4, A3, A2 -> A0,A1,A2
A4, A3, A3 -> A0,A1,A2,A3
A4, A3, A4 -> A2,A3
A4, A4, A2 -> A3,A4
A4, A4, A3 -> A3,A4

Using the models

In [15]:
model1.predict(enrollments)
Out[15]:
[14923.233333333337,
 15319.950000000004,
 15319.950000000004,
 15716.666666666672,
 16113.383333333339,
 16113.383333333339,
 17700.250000000007,
 17303.53333333334,
 15716.666666666672,
 15319.950000000004,
 15319.950000000004,
 15716.666666666672,
 15716.666666666672,
 16113.383333333339,
 17700.250000000007,
 19287.116666666676,
 19287.116666666676,
 19287.116666666676,
 18890.40000000001,
 18493.68333333334]
In [16]:
model2.predict(enrollments)
Out[16]:
[13435.14,
 14264.14,
 15028.14,
 15126.2,
 15418.2,
 15429.14,
 16375.14,
 16734.2,
 16944.38,
 15248.2,
 15806.32,
 14960.2,
 14978.2,
 15305.08,
 16427.14,
 17471.08,
 18785.2,
 19143.2,
 19399.260000000002,
 18691.2]

Comparing the models

In [17]:
bchmk.plot_compared_series(enrollments, [model1, model2], bchmk.colors, intervals=False)
In [18]:
bchmk.print_point_statistics(enrollments, [model1, model2])

Model		& Order     & RMSE		& SMAPE      & Theil's U		\\ 
HOFTSFTS		& 3		& 426.81		& 0.99		& 0.7	\\ 
HOFTSFTS Diff		& 3		& 970.87		& 2.53		& 1.58	\\

Residual Analysis

In [12]:
from pyFTS.benchmarks import ResidualAnalysis as ra

ra.plot_residuals(enrollments, [model1, model2])

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/usr/lib/python3/dist-packages/IPython/core/formatters.py in __call__(self, obj)
    305                 pass
    306             else:
--> 307                 return printer(obj)
    308             # Finally look for special method names
    309             method = get_real_method(obj, self.print_method)

/usr/lib/python3/dist-packages/IPython/core/pylabtools.py in <lambda>(fig)
    225 
    226     if 'png' in formats:
--> 227         png_formatter.for_type(Figure, lambda fig: print_figure(fig, 'png', **kwargs))
    228     if 'retina' in formats or 'png2x' in formats:
    229         png_formatter.for_type(Figure, lambda fig: retina_figure(fig, **kwargs))

/usr/lib/python3/dist-packages/IPython/core/pylabtools.py in print_figure(fig, fmt, bbox_inches, **kwargs)
    117 
    118     bytes_io = BytesIO()
--> 119     fig.canvas.print_figure(bytes_io, **kw)
    120     data = bytes_io.getvalue()
    121     if fmt == 'svg':

/usr/local/lib/python3.6/dist-packages/matplotlib/backend_bases.py in print_figure(self, filename, dpi, facecolor, edgecolor, orientation, format, **kwargs)
   2214                     orientation=orientation,
   2215                     dryrun=True,
-> 2216                     **kwargs)
   2217                 renderer = self.figure._cachedRenderer
   2218                 bbox_inches = self.figure.get_tightbbox(renderer)

/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_agg.py in print_png(self, filename_or_obj, *args, **kwargs)
    505 
    506     def print_png(self, filename_or_obj, *args, **kwargs):
--> 507         FigureCanvasAgg.draw(self)
    508         renderer = self.get_renderer()
    509         original_dpi = renderer.dpi

/usr/local/lib/python3.6/dist-packages/matplotlib/backends/backend_agg.py in draw(self)
    428             # if toolbar:
    429             #     toolbar.set_cursor(cursors.WAIT)
--> 430             self.figure.draw(self.renderer)
    431         finally:
    432             # if toolbar:

/usr/local/lib/python3.6/dist-packages/matplotlib/artist.py in draw_wrapper(artist, renderer, *args, **kwargs)
     53                 renderer.start_filter()
     54 
---> 55             return draw(artist, renderer, *args, **kwargs)
     56         finally:
     57             if artist.get_agg_filter() is not None:

/usr/local/lib/python3.6/dist-packages/matplotlib/figure.py in draw(self, renderer)
   1297 
   1298             mimage._draw_list_compositing_images(
-> 1299                 renderer, self, artists, self.suppressComposite)
   1300 
   1301             renderer.close_group('figure')

/usr/local/lib/python3.6/dist-packages/matplotlib/image.py in _draw_list_compositing_images(renderer, parent, artists, suppress_composite)
    136     if not_composite or not has_images:
    137         for a in artists:
--> 138             a.draw(renderer)
    139     else:
    140         # Composite any adjacent images together

/usr/local/lib/python3.6/dist-packages/matplotlib/artist.py in draw_wrapper(artist, renderer, *args, **kwargs)
     53                 renderer.start_filter()
     54 
---> 55             return draw(artist, renderer, *args, **kwargs)
     56         finally:
     57             if artist.get_agg_filter() is not None:

/usr/local/lib/python3.6/dist-packages/matplotlib/axes/_base.py in draw(self, renderer, inframe)
   2435             renderer.stop_rasterizing()
   2436 
-> 2437         mimage._draw_list_compositing_images(renderer, self, artists)
   2438 
   2439         renderer.close_group('axes')

/usr/local/lib/python3.6/dist-packages/matplotlib/image.py in _draw_list_compositing_images(renderer, parent, artists, suppress_composite)
    136     if not_composite or not has_images:
    137         for a in artists:
--> 138             a.draw(renderer)
    139     else:
    140         # Composite any adjacent images together

/usr/local/lib/python3.6/dist-packages/matplotlib/artist.py in draw_wrapper(artist, renderer, *args, **kwargs)
     53                 renderer.start_filter()
     54 
---> 55             return draw(artist, renderer, *args, **kwargs)
     56         finally:
     57             if artist.get_agg_filter() is not None:

/usr/local/lib/python3.6/dist-packages/matplotlib/axis.py in draw(self, renderer, *args, **kwargs)
   1131         renderer.open_group(__name__)
   1132 
-> 1133         ticks_to_draw = self._update_ticks(renderer)
   1134         ticklabelBoxes, ticklabelBoxes2 = self._get_tick_bboxes(ticks_to_draw,
   1135                                                                 renderer)

/usr/local/lib/python3.6/dist-packages/matplotlib/axis.py in _update_ticks(self, renderer)
    972 
    973         interval = self.get_view_interval()
--> 974         tick_tups = list(self.iter_ticks())
    975         if self._smart_bounds and tick_tups:
    976             # handle inverted limits

/usr/local/lib/python3.6/dist-packages/matplotlib/axis.py in iter_ticks(self)
    915         Iterate through all of the major and minor ticks.
    916         """
--> 917         majorLocs = self.major.locator()
    918         majorTicks = self.get_major_ticks(len(majorLocs))
    919         self.major.formatter.set_locs(majorLocs)

/usr/local/lib/python3.6/dist-packages/matplotlib/ticker.py in __call__(self)
   1951     def __call__(self):
   1952         vmin, vmax = self.axis.get_view_interval()
-> 1953         return self.tick_values(vmin, vmax)
   1954 
   1955     def tick_values(self, vmin, vmax):

/usr/local/lib/python3.6/dist-packages/matplotlib/ticker.py in tick_values(self, vmin, vmax)
   1959         vmin, vmax = mtransforms.nonsingular(
   1960             vmin, vmax, expander=1e-13, tiny=1e-14)
-> 1961         locs = self._raw_ticks(vmin, vmax)
   1962 
   1963         prune = self._prune

/usr/local/lib/python3.6/dist-packages/matplotlib/ticker.py in _raw_ticks(self, vmin, vmax)
   1901         if self._nbins == 'auto':
   1902             if self.axis is not None:
-> 1903                 nbins = np.clip(self.axis.get_tick_space(),
   1904                                 max(1, self._min_n_ticks - 1), 9)
   1905             else:

/usr/local/lib/python3.6/dist-packages/matplotlib/axis.py in get_tick_space(self)
   2060         size = tick.label1.get_size() * 3
   2061         if size > 0:
-> 2062             return int(np.floor(length / size))
   2063         else:
   2064             return 2**31 - 1

ValueError: cannot convert float NaN to integer
<matplotlib.figure.Figure at 0x7f5a151d6f28>
In [ ]:
In [ ]: