@@ -20,40 +20,40 @@ The software architecture deals with abstraction, with decomposition and composi
### The development view of PySD
To represent the architecture of the fragments that make up *PySD* a package diagram is used, represented in [PySD development view]. In this diagram, the hierarchical architecture of pysd packages is represented, excluding the relationship between their modules. In [PySD relationships between modules] we can find the relationships between *PySD* modules.
A package diagram, represented in the figure [PySD development view], is used to represent the architecture of the fragments that make up *PySD*. This diagram represents the hierarchical architecture of PySD packages, excluding the relationship between their modules. In [PySD relationships between modules] we can find the relationships between *PySD* modules.
![PySD development view]
**pysd** is the main package that includes all the others and all the system logic. It is necessary to differentiate between package called **pysd** and module called pysd. The latter allows the user to interact with the system and has adequate functions to be able to translate a Vensim or XMILE model, and, in addition, to execute the generated translation. However, the XMILE translation process is not defined in this document.
**pysd** is the main package that includes all the others and all the logic of the system. It is necessary to differentiate between the package called **pysd** and the module called pysd. The latter allows the user to interact with the system and has the appropiate functions to be able to translate a Vensim or XMILE model and, in addition, to execute the generated translation. However, the XMILE translation process is not defined in this document.
**pysd** module interacts with modules in the **py_backend** package. This package has two sub-packages: **vensim** and **xmile**. **Vensim** package contains all the needed logic to translate *Vensim* models.
The **pysd** module interacts with the modules of the **py_backend** package. This package has two sub-packages: **vensim** and **xmile**. The **vensim** package contains all the logic needed to translate *Vensim* models.
In [PySD relationships between modules] the relationships between the main modules are represented. For clarity, each module has been represented with its name package.
In [PySD relationships between modules] the relationships between the main modules are represented. For clarity, each module is represented with its name package.
![PySD relationships between modules]
As previously discussed, users interact with **pysd** module. In turn, **pysd**takes care of translating the *Vensim* model to the *Python* model by interacting with **vensim2py**, which creates the correct *Python* translation. In this process, **vensim2py** interacts and uses the functions of the modules: **external**, **utils** and **builder**. To carry out the execution process, **pysd** uses the **functions** module.
As mentioned above, users interact with **pysd** module. In turn, **pysd** is responsible for translating the *Vensim* model into the *Python* model by interacting with **vensim2py**, which creates the correct *Python* translation. In this process, **vensim2py** interacts with and uses the functions of the modules: **external**, **utils** and **builder**. To carry out the execution process, **pysd** uses the **functions** module.
## The logical view of PySD
The purpose of each PySD module is detailed below as well as the most important functions of each module.
It should be noted that in diagrams it has been necessary, it is detailed input parameters with *in*notation, output parameters with *out* notation and parameters that are modified in a function with *inout* notation. In addition, the different types of parameters that could be, are described through notes in convenient diagrams, which is due to Python's dynamic typing.
The purpose of each PySD module is detailed below, as well as the most important functions of each module.
It should be noted that in diagrams it has been necessary, input parameters have been detailed with *in* notation, output parameters with *out* notation and parameters that are modified in a function with *inout* notation. In addition, the different types of parameters that could be, are described by notes in convenient diagrams, which is due to Python's dynamic typing.
In [Main modules of PySD]: **pysd**, **vensim2py** and **table2py** modules are presented with detail. The **pysd** module is in package *pysd*, while **vensim2py** and **table2py** module, in package *vensim*.
In [Main modules of PySD]: **pysd**, **vensim2py** and **table2py** modules are presented in detail. The **pysd** module is in the *pysd* package, while the **vensim2py** and **table2py** modules are in the *vensim* package.
The **pysd** module has the necessary functions to allow the user to create the translation of a Vensim model to Python. The function **read\_vensim**takes as a parameter a Vensim model in text format and converts it to an instance of the Model class, which is in **functions** module. In the same way, **pysd** has **load** function, which can generate from a Python model to an instance of *Model* class, which can be able to execute and perform the simulation. The **load** function is used within**read\_ vensim** function.
The **pysd** module has the necessary functions to allow the user to create the translation of a Vensim model into Python. The **read\_vensim**function takes a Vensim model in text format as a parameter and converts it into an instance of the Model class, which is in the **functions** module. Also, **pysd** has the **load** function, which can generate from a Python model to an instance of the *Model* class, which may be able to execute and perform the simulation. The **load** function is used inside the**read\_ vensim** function.
![Main modules of PySD]
The table2py module has only one function, **read_tabular**. This function allows to read a Vensim model in table form (csv, tab or xlsx) and convert it into an instance of Model class.
The table2py module has only one function, **read_tabular**. This function allows to read a Vensim model in table form (csv, tab or xlsx) and convert it into an instance of the Model class.
In addition, **vensim2py** is represented in that diagram. In **vensim2py**, the five grammars of *PySD* are defined, with their associated classes which allows parser and get the information from a Vensim model.
In addition, **vensim2py** is represented in that diagram. In **vensim2py**, the five grammars of *PySD* are defined, with their associated classes that allow parsing and obtaining the information from a Vensim model.
The main function of the *vensim2py* module, which is also used by **read\_vensim** function of *pysd* module, is **translate\_vensim**. This function starts the translation process. The Vensim model is parsed with the first *pysd* grammar, *file\_structure\_grammar*, found within**get\_file\_sections** function. The *file\_structure\_grammar* divides the model into sections: the main section with the main code of the model and on the other hand, a section for each macro in the model. The obtained sections are passed as parameters to **translate\_section** function later.
The main function of the *vensim2py* module, which is also used by the **read\_vensim** function of *pysd* module, is **translate\_vensim**. This function starts the translation process. The Vensim model is parsed with the first *pysd* grammar, *file\_structure\_grammar*, found in the**get\_file\_sections** function. The *file\_structure\_grammar* divides the model into sections: the main section with the main code of the model and on the other hand, a section for each macro in the model. The obtained sections are passed as parameters to **translate\_section** function afterwards.
The **get\_model\_elements**, **get\_equation\_components**, **parse\_general\_expression** and **parse\_lookup\_expression**functions have the four remaining grammars of *PySD* which are: *model\_structure\_grammar*, *component\_structure\_grammar*, *expression\_grammar* and *lookup\_grammar*, respectively. Moreover, after each of these functions, the NodeVisitor classes that are associated with each grammar are defined. These classes allow to perform and parse the parse tree.
The functions **get\_model\_elements**, **get\_equation\_components**, **parse\_general\_expression** and **parse\_lookup\_expression** have the four remaining grammars of *PySD* which are: *model\_structure\_grammar*, *component\_structure\_grammar*, *expression\_grammar* and *lookup\_grammar*, respectively. In addition, after each of these functions, the NodeVisitor classes associated with each grammar are defined. These classes allow the parse tree to be performed and parsed.
Noteworthy is the **\_include\_common\_grammar**function which has the basic grammar rules used by all other grammars.
Noteworthy is the function **\_include\_common\_grammar** which has the basic grammar rules used by all other grammars.
Due to the complexity of vensim2py, since it has the five functions in which PySD grammars and their visitor classes are defined, in [Simplified vensim2py module] it is represented without detail. These classes are: FileParser, ModelParser, ComponentParser, ExpressionParser and LookupParser. Note that these classes inherit from the NodeVisitor class, that provides an inversion-of-control framework for walking a tree and returning a new construct based on it.
Due to the complexity of vensim2py, as it has the five functions in which PySD grammars and their visitor classes are defined, in [Simplified vensim2py module] it is represented without detail. These classes are: FileParser, ModelParser, ComponentParser, ExpressionParser and LookupParser. Note that these classes inherit from the NodeVisitor class, that provides an inversion-of-control framework for walking a tree and returning a new construct based on it.
![Simplified vensim2py module]
...
...
@@ -61,62 +61,64 @@ In [Classes of pysd grammars] and [Classes of pysd grammars2] are represented th
![Classes of pysd grammars]
![Classes of pysd grammars2]
The methods of each class are the visitor methods associated with the different grammar rules. There is not a visitor method for each rule, but there is a visitor method associated with a rule that serves to store certain information about the parsed model. Within the visitor method, that relevant information is stored in the attributes of each class, which are then returned as a result of the grammar.
Visitor methods always have three parameters: *self*, *n* and *vc*. *Self* represents the current instance of the class, *n* is of the type Node and is the node being visited, and *vc* or *visit children* is a list of all the results from the child nodes of that expression being parsed. From that last parameter, vc, the information is taken and saved in the attributes of the classes.
The methods of each class are the visitor methods associated with the different grammar rules. There is no visitor method for each rule, but there is a visitor method associated with a rule that serves to store certain information about the parsed model. Within the visitor method, that relevant information is stored in the attributes of each class, which are then returned as a result of the grammar.
The **functions** module is represented in [Functions module]. It is one of the most important modules in PySD, since it has the classes that will instantiate the Python translation model and also has the necessary logic to execute the simulation. In that diagram, the classes it has and the relationships between them are represented.
Visitor methods always have three parameters: *self*, *n* and *vc*. *Self* represents the current instance of the class, *n* is of type Node and is the node being visited, and *vc* or *visit children* is a list of all the results of the child nodes of the expression being parsed. From that last parameter, vc, the information is taken and stored in the attributes of the classes.
The **functions** module is represented in [Simplified functions module]. It is one of the most important modules in PySD, since it has the classes that will instantiate the Python translation model and also has the logic needed to run the simulation. That diagram represents the classes it has and the relationships between them.
![Simplified functions module]
The **functions** module with details is found in the diagram of [Functions module] as well as the **Time** class that is define in this module. In **functions**, we can find many functions that are used in Vensim but with the pertinent logic in Python, for example: PULSE, IF THEN ELSE, RANDOM UNIFORM, etc.
The **functions** module in detail can be found in the [Functions module (Part 1)] diagram as well as the **Time** class that is define in this module. In **functions**, we can find many functions that are used in Vensim but with the relevant logic in Python, for example: PULSE, IF THEN ELSE, RANDOM UNIFORM, etc.
The **Time** class represents the time throughout the simulation. The *t* attribute represents the current time, which changes as the simulation progresses, and the *step* attribute represents the time increment that occurs in each iteration.
The **Time** class represents the time throughout the simulation. With the *t* attribute represents the current time, which is modified according as the simulation moves, and the *step* attribute the time increment that occurs in each iteration.
![Functions module]
![Functions module (Part 1)]
In the diagram [Functions module2] the classes of the functions module **Stateful**, **Integ**, **Macro** and **Model** are represented in details. The **Stateful** class is one of the most relevant classes of that module, since, except Time, all of the classes inherit from it. This class allows to represents the state evolution of a certain element models, recreating the simulation process in Vensim. To do this, it has an attributed called *state* that simulates the state of the elements and changes its value in each iteration of the simulation.
In the diagram [Functions module (Part 2)] the classes of the functions module **Stateful**, **Integ**, **Macro** and **Model** are represented in detail. The **Stateful** class is one of the most relevant classes of that module, since, except Time, all the classes inherit from it. This class makes it possible to represents the evolution of the state of a certain element models, recreating the simulation process in Vensim. To do so, it has an attributed called *state* that simulates the state of the elements and changes its value in each iteration of the simulation.
<!--Cambiar nombres-->
![Functions module2]
![Functions module (Part 2)]
The **Integ** class simulates the Vensim stocks. It receives and stores an initial value and has the function from which the derivative necessary to carry out the integration is obtained.
The **Integ** class simulates the Vensim stocks. It receives and stores an initial value and has the function from which the derivative necessary to perform the integration is obtained.
The Model class stores all the information about the main code of the model (translated). An instance of this class is called a pysd model, it is the Python language representation of the Vensim file. That is, the Model class implements a representation of the model's stateful elements and has most of the methods for being able to access and modify the model's components. In addition, the Model class is in charge of instantiating time as a function of the model variables and is also in charge of carrying out the simulation through Euler integration.
The Model class stores all the information about the main code of the (translated) model. An instance of this class is called a pysd model, it is the Python language representation of the Vensim file. That is, the Model class implements a representation of the stateful elements of the model and has most of the methods to access and modify the components of the model. In addition, the Model class is in charge of instantiating the time as a function of the model variables and it is also in charge of performing the simulation through Euler integration.
The **initialize** function of that class initialize the model simulation. The **run** function allows to simulate the behaviour of the model through increasing the steps. And the **\_euler\_step** function allows to do the Euler integration in a single step, using the state of the Stateful elements and updating it.
The **initialize** function of that class initialize the model simulation. The **run** function allows to simulate the behaviour of the model by increasing steps. And the **\_euler\_step** function allows to do the Euler integration in a single step, using the state of the Stateful elements and updating it.
The **Model** class inherits from Macro class. The logic for rendering Vensim macros is implemented in Macro class. This class gets the stateful objects which had been created in translation process and they are initialized to later obtain their derivates and the results of the execution. Model does the same functions of Macro, but Model is the root model object so it has more methods to facilitate execution.
The **Model** class inherits from Macro class. The logic for rendering Vensim macros is implemented in Macro class. This class obtains the stateful objects that have been created in the translation process and they are initialized to later obtain their derivates and the results of the execution. Model does the same functions as Macro, but Model is the root model object so it has more methods to facilitate execution.
Next, in [Builder module] figures the **builder** module. There is no class defined in this module, but it is charge in making the text model in Python, using the results obtained in the translation. It has the necessary code to assemble in a model pysd all the elements of both Vensim or XMILE and make, from these, a versioncompatible with Python.
Next, in [Builder module] figures the **builder** module. There is no class defined in this module, but it is in charge of making the text model in Python, using the results obtained in the translation. It has the necessary code to assemble in a pysd model all the elements of both Vensim or XMILE and make, from these, a Python-compatible version.
![Builder module]
The main function of the builder module is **build**. That function builds and writes the Python representation of the model. It is called from vensim2py module after finishing the entire translation process of the Vensim model. As parameters are passed the different elements of the model that have been parsed, subscripts, namespace and the name of the file where the result of the representation in Python must be written. This function has certain lines of permanent code that always write in the created models, but then, there are certain lines of code that are completed with the translation generated before in the vensim2py model.
The main function of the builder module is **build**. That function builds and writes the Python representation of the model. It is called from the vensim2py module after finishing the whole process of translating the Vensim model. As parameters it is passed the different elements of the model that have been parsed, subscripts, namespace and the name of the file where the result of the Python representation should be written. This function has certain permanent lines of code that are always write in the models created, but then, there are certain lines of code that are completed with the translation generated before in the vensim2py model.
In image [Utils module] is found the utils module. The main purpose of utils is to join in a single module all the functions with great utility for the project. Many of these functions are used many times throughout the translation process. So, as we already presented in [PySD relationships between modules], this module is used by builder, functions, external and vensim2py modules. In turn, the accesible names of the decorators, external and functions modules are imported into the utils modules to define a list with the names that hace already been used and that have a particular meaning in the model that is being translated.
In image [Utils module] is found the utils module. The main purpose of utils is to bring together in a single module all the functions that are useful for the project. Many of these functions are used many times during the translation process. So, as already presented in [PySD relationships between modules], this module is used by the builder, functions, external and vensim2py modules. In turn, the accesible names of the decorators, external and functions modules are imported into the utils modules to define a list of the names that have already been used and that have a particular meaning in the model being translated.
![Utils module]
[Simplified external module] represents the external module and the classes it contains without detail. The main purpose of the classes that are defined in that module is to read external data. The main objective of the external module is to join in a single file, all the required functions or tools to read external data files.
[Simplified external module] represents the external module and the classes it contains without detail. The main purpose of the classes defined in that module is to read external data. The main objective of the external module is to gather in a single file, all the required functions or tools to read external data files.
![Simplified external module]
The figure [External module] shows the detailed diagrams of the External and Excels class.
External is the main class of that module, all of the other classes inherit from it, except the Excels class.
The figure [External module (Part 1)] shows the detailed diagrams of the External and Excels class.
External is the main class of that module, all other classes inherit from it, except the Excels class.
![External module]
![External module (Part 1)]
The External class allows to store certain information, such as the name of the file being read and the data it contains.
The External class allows storing certain information, such as the name of the file being read and the data it contains.
The Excels class is in charge of reading Excel files and storing information about them, to avoid reading these files more than once, putting the singleton pattern in to practice.
The Excels class is in charge of reading Excel files and storing information about them, in order to avoid reading these files more than once, implementing the singleton pattern.
In [External module2] all the classes that inherit from the External class are presented.
In [External module (Part 2)] all the classes that inherit from the External class are presented.
![External module2]
![External module (Part 2)]
In Vensim there are different statements that allow to obtain data from external files that are used as variables in a Vensim model. The set of these functions that are supported in PySD is presented below.
In Vensim there are different statements that allow to obtain data from external files that are used as variables in a Vensim model. Below is the set of these functions that are supported in PySD.
To obtain the data from statements like GET XLS DATA and GET DIRECT DATA, the ExtData class is found. In turn, to the GET XLS LOOKUPS and GET DIRECT LOOKUPS statements, the ExtLookup class. For GET XLS CONSTANT and GET DIRECT CONSTANT functions, the ExtConstant class and, finally, to implement the GET XLS SUBSCRIPT and GET DIRECT SUBSCRIPT function, it is the ExtSubscript class.
To obtain data from statements like GET XLS DATA and GET DIRECT DATA, there is the ExtData class. In turn, for the GET XLS LOOKUPS and GET DIRECT LOOKUPS statements, the ExtLookup class. For the GET XLS CONSTANT and GET DIRECT CONSTANT functions, the ExtConstant class and, finally, to implement the GET XLS SUBSCRIPT and GET DIRECT SUBSCRIPT function, the ExtSubscript class.
These expressions create a new instance of the External class where information is stored to represent the necessary data structures. These instances of the External class are initialized before stateful objects.
...
...
@@ -190,13 +192,13 @@ Two main scenarios can be distinguished throughout the
[Classes of pysd grammars]:/images/logical-view/grammar1.png"Classes of pysd grammars"
[Classes of pysd grammars2]:/images/logical-view/grammar2.png"Classes of pysd grammars"
