pyFTS.distributed package

Module contents

Submodules

pyFTS.distributed.dispy module

pyFTS.distributed.spark module

pyFTS.distributed.spark.create_multivariate_model(**parameters)

From the dictionary of parameters, create a multivariate FTS model

Parameters:parameters – dictionary of parameters
Returns:multivariate FTS model
pyFTS.distributed.spark.create_spark_conf(**kwargs)

Configure the Spark master node

Parameters:kwargs –
Returns:
pyFTS.distributed.spark.create_univariate_model(**parameters)

From the dictionary of parameters, create an univariate FTS model

Parameters:parameters – dictionary of parameters
Returns:univariate FTS model
pyFTS.distributed.spark.distributed_predict(data, model, **kwargs)

The main method for distributed forecasting with FTS models using Spark clusters.

It takes a trained FTS model and the test data, connect with the Spark cluster, proceed the distributed forecasting and return the merged forecasted values.

Parameters:
  • model – an FTS trained model
  • data – test data
  • url – URL of the Spark master
  • app –
Returns:

forecasted values
pyFTS.distributed.spark.distributed_train(model, data, **kwargs)

The main method for distributed training of FTS models using Spark clusters.

It takes an empty model and the train data, connect with the Spark cluster, proceed the distributed training and return the learned model.

Parameters:
  • model – An empty (non-trained) FTS model
  • data – train data
  • url – URL of the Spark master node
  • app – Application name
Returns:

trained model
pyFTS.distributed.spark.get_clustered_partitioner(explanatory_variables, target_variable, **parameters)

Return the UoD partitioner from the ‘shared_partitioner’ fuzzy sets, special case for clustered multivariate FTS.

Parameters:
  • explanatory_variables – the list with the names of the explanatory variables
  • target_variable – the name of the target variable
Returns:

Partitioner object
pyFTS.distributed.spark.get_partitioner(shared_partitioner, type='common', variables=[])

Return the UoD partitioner from the ‘shared_partitioner’ fuzzy sets

Parameters:
  • shared_partitioner – the shared variable with the fuzzy sets
  • type – the type of the partitioner
  • variables – in case of a Multivariate FTS, the list of variables
Returns:

Partitioner object
pyFTS.distributed.spark.get_variables(**parameters)

From the dictionary of parameters, return a tuple with the list of explanatory and target variables

Parameters:parameters – dictionary of parameters
Returns:a tuple with the list of explanatory and target variables
pyFTS.distributed.spark.share_parameters(model, context, data)

Create a shared variable with a dictionary of the model parameters and hyperparameters

Parameters:
  • model – the FTS model to extract the parameters and hyperparameters
  • context – Spark context
  • data – dataset
Returns:

the shared variable with the dictionary of parameters
pyFTS.distributed.spark.slave_forecast_multivariate(data, **parameters)

Receive test data, create a multivariate FTS model from the parameters and return the forecasted values

Parameters:
  • data – test data
  • parameters – dictionary of parameters
Returns:

forecasted values from the data input
pyFTS.distributed.spark.slave_forecast_univariate(data, **parameters)

Receive test data, create an univariate FTS model from the parameters and return the forecasted values

Parameters:
  • data – test data
  • parameters – dictionary of parameters
Returns:

forecasted values from the data input
pyFTS.distributed.spark.slave_train_multivariate(data, **parameters)

Receive train data, train a multivariate FTS model and return the learned rules

Parameters:
  • data – train data
  • parameters – dictionary of parameters
Returns:

Key/value list of the learned rules
pyFTS.distributed.spark.slave_train_univariate(data, **parameters)

Receive train data, train an univariate FTS model and return the learned rules

Parameters:
  • data – train data
  • parameters – dictionary of parameters
Returns:

Key/value list of the learned rules