Ant

Note

This guide requires basic knowledge of Apache Ant.

Quick start

👷👷🏾

How it works

Ant is used as an orchestrator to manage all the actions on the Trenissimo projects, such as model validation, timetable validation, preparation and simulation executions, output file generation, external tools interaction, etc. On the GUI side, the available project actions are Clean, Build, Run, and Test. Each of these is implemented through a target Ant using a corresponding name, defined in the build file nbproject/build-impl.xml. These files are automatically updated at each Trenissimo release. Any changes to the nbproject/build-impl.xml file will be overwritten consequently. Every change to the build process described in this guide should be written to the build.xml file, which is not affected by the automatic updates.

All Trenissimo specific tasks are contained in the com.trenolab.trilogy.ant package, with the following namespace antlib:com.trenolab.trilogy.ant and usually with trilogy prefix.

When the user executes one of the GUI actions mentioned above or calls the corresponding target from the CLI, Ant firstly executes the top-level tasks present in the file build.xml. One of the tasks is the import of nbproject/build-impl.xml, which triggers the execution of all the top-level tasks present in nbproject/build-impl.xml. Then, depending on the requested target, Ant constructs the dependency tree, based on the depends attributes (in the target element) and on the extension-point.

Ant script customizations rely on the extension-point.

Extension points

Extension-Points are similar to targets in that they have a name and a depends list and can be executed from the command line. Just like targets they represent a state during the build process.

Unlike targets they don't contain any tasks, their main purpose is to collect targets that contribute to the desired state in their depends list.

Targets can add themselves to an extension-point's depends list via their extensionOf attribute. The targets that add themselves will be added after the targets of the explicit depends attribute of the extension-point, if multiple targets add themselves, their relative order is not defined.

The main purpose of an extension-point is to act as an extension point for build files designed to be imported. In the imported file, an extension-point defines a state that must be reached and targets from other build files can join the depends list of said extension-point in order to contribute to that state.

From Apache Ant

Phases

For simplicity, we can split the execution of each action into several phases, which group together all the needed targets to perform the actions.

Except init, which is automatically invoked, it is possible to invoke all the other phases from an action through the GUI, from the project's context menu, from Trenissimo's main toolbar, or from the CLI. Furthermore, the simulation projects have a Clean and Build action which calls in sequence the Clean and Build phases.

Init

During the init phase, the configuration files are read, and the execution environment common to the other phases is prepared. This phase is essential and is called automatically before any other phase. There is no action corresponding to the init phase.

Clean

The task of this phase is to eliminate any temporary output files related to the projects. It is invoked by the Clean action.

Build

In this phase, the project files are read and analyzed to find possible errors and inconsistencies. Also, the simulation projects create some temporary files which are needed for the simulation execution and, if enabled, a Simulation Archive file is created which contains all that is needed to execute the simulation from the command line or from the Simulation Launcher.

Run

Available only for the projects of type Simulation, Stochastic Simulation, Scripted simulation and Simulation variant. This is the phase that executes the simulation. If prior to calling the Run action on a project, the Build phase was not executed, it will be done automatically. The run phase is invoked by the Run action.

Test

Available only for the projects of type Simulation. This phase executes an _asynchronous simulation, which means that it simulates the entire path for each train in an independent and isolated manner, neglecting the conflicts and the interactions between the trains. As output, this phase produces a CSV file containing the delay to the destination of each train, useful to spot if any trains are not able to fulfill the planned timetable constraints even in absence of conflicts.

Extension points

The Extension Points are used for adding tasks to be executed before, after or during some phases of the project's simulation, taken from the build.xml file, this way the nbproject/build-impl.xml file does not require any changes.

Not all project types and phases have an extension point; let's look at the most useful ones.

Simulation projects and Variant simulation projects

• init phase: called right after the main init.

  <target name="my-init" extensionOf="init">
      ...
  </target>

• clean phase: called right after the main clean.

  <target name="my-clean" extensionOf="clean">
      ...
  </target>

• build phase: called right after the main build.

  <target name="my-build" extensionOf="build">
      ...
  </target>

• configure-build phase: called right before the simulation build. This is the best phase to configure the build, for example using the tasks trilogy:timetableTransformationsConfiguration and trilogy:simulationTimetableTransformationsConfiguration. If the clean phase was executed on the simulation, none of the build's artifacts will be present in this phase.

  <target name="my-configure-build" extensionOf="configure-build">
       <trilogy:timetableTransformationsConfiguration>
          <my:applyTimetableTransformations/>
      </trilogy:timetableTransformationsConfiguration>
  </target>

• configure phase: called right before the execution of the simulation. In this phase we have the certainty that the pre-run phases of the plugins were already called. This is the best phase to configure the simulator, for example with the tasks trilogy:groovySimulationListenersConfiguration and trilogy:deterministicSimulatorConfiguration.

  <target name="my-configure" extensionOf="configure">
      <trilogy:groovySimulationListenersConfiguration scriptFile="my-script.listener">
          <property name="baseDir" value="${basedir}"/>
          <reference name="settings" reference="ref.settings"/>
      </trilogy:groovySimulationListenersConfiguration>
      <trilogy:deterministicSimulatorConfiguration>
          <my:simulationContext/>
          <my:simulationOutput outputFile="${params.outputDir}/my-output.csv"/>
      </trilogy:deterministicSimulatorConfiguration>        
  </target>

• pre-run phase: called before configure, inside the run phase.

  <target name="my-pre-run" extensionOf="pre-run">
      ...
  </target>

• pre-pre-run phase: called right before pre-run, inside the main run phase. Can be used to prepare the needed resources for the pre-run of several plugins, in order to guarantee the order of execution.

  <target name="my-pre-pre-run" extensionOf="pre-pre-run">
      ...
  </target>

• post-run phase: called right after the execution of the simulation, inside the main run phase.

  <target name="my-post-run" extensionOf="post-run">
      ...
  </target>

• post-post-run phase: called right after the post-run phase, inside the main run phase. Can be used to close resources that several plugins needed till the post-run, in order to guarantee the order of execution.

  <target name="my-post-post-run" extensionOf="post-post-run">
      ...
  </target>

Stochastic simulation projects

• init phase: called right after the main init.

  <target name="my-init" extensionOf="init">
      ...
  </target>

• clean phase: Called right after the main clean.

  <target name="my-clean" extensionOf="clean">
      ...
  </target>

• build phase: called right after the main biuld.

  <target name="my-build" extensionOf="build">
      ...
  </target>

• configure-build phase: called right before the build of the simulation. This is the best phase to configure the build, for example with the tasks trilogy:timetableTransformationsConfiguration and trilogy:simulationTimetableTransformationsConfiguration. If the clean phase of the simulation was executed, none of the build artifacts will be present in this phase.

  <target name="my-configure-build" extensionOf="configure-build">
       <trilogy:timetableTransformationsConfiguration>
          <my:appltTimetableTransformations/>
      </trilogy:timetableTransformationsConfiguration>
  </target>

• configure-stochastic phase: called right before the execution of the stochastic simulation. This is the best phase to configure the simulator, for example with the task trilogy:stochasticSimulationConfiguration.

  <target name="my-configure" extensionOf="configure-stochastic">
      <trilogy:stochasticSimulationConfiguration runKeyParam="runKey" runSimulationIdParam="runSimulationId" configurationReferenceParam="configurationReference">
          <initializer>
              <trilogy:groovySimulationListenersConfiguration  configurationReference="@{configurationReference}" scriptFile="lib/my-script.listener">
                  <property name="baseDir" value="${basedir}"/>
                  <reference name="settings" reference="ref.settings"/>
              </trilogy:groovySimulationListenersConfiguration>
              <trilogy:deterministicSimulatorConfiguration>
                  <my:simulationContext/>
                  <my:simulationOutput outputFile="${params.outputDir}/my-output.csv"/>
              </trilogy:deterministicSimulatorConfiguration>
          </initializer>
      </trilogy:stochasticSimulationConfiguration>     
  </target>

• process-stochastic-results phase: This phase is enabled only if one of the outputs for TRENOanalysis or for Sim Analyzer is enabled, or if in configure-stochastic the property stochasticresults.enabled was set. In this extension point, the ref.stochasticData reference is present of type com.trenolab.trilogy.simulation.stochastic.SimulationResults containing the planned and simulation times for each train.

  <target name="my-process-stochastic-results" extensionOf="process-stochastic-results">
      <trilogy:groovyScript>
          <fileset dir="${basedir}" includes="lib/myProcess.groovy"/>
          <property name="outputDir" location="${params.outputDir}"/>
          <reference name="stochasticData" ref="ref.stochasticData"/>
      </trilogy:groovyScript>
      <my:processResults outputDir="${params.outputDir}" inputReference="ref.stochasticData">
  </target>

• post-run phase: called at the end of the stochastic simulation. Can be used to aggregate the output files of the single runs through a groovy script or a task. It's equivalent to the scripts of type post-run (extension .postrun)

  <target name="my-post-run" extensionOf="post-run">
      <trilogy:groovyScript>
          <fileset dir="${basedir}" includes="lib/myPostrun.groovy"/>
          <property name="outputDir" location="${params.outputDir}"/>
      </trilogy:groovyScript>
      <my:processOutputFiles dir="${params.outputDir}" >
  </target>

Global listeners configuration

In the configure and the configure-stochastic phases, a plugin can define a global configuration, shared to all other plugins and simulation listeners, with the trilogy:groovySimulationListenersConfiguration task with global attribute set to true. One use of this feature could be adding the module classpath to all the listeners, defining implicit imports, setting references, or defining shared functions.

<trilogy:groovySimulationListenersConfiguration global="true">
    <!-- add the powermodelling plugin to all the listeners classpath -->
    <classpathModule module="com.trenolab.trilogy.powermodelling"/>

    <!-- add a new implicit import to all the listeners -->
    <groovyImport type="star" package="com.trenolab.trilogy.powermodelling.model.events"/>

    <!-- set a reference -->
    <reference name="powerModellingConfiguration" reference="ref.powermodelling.configuration"/>
</trilogy:groovySimulationListenersConfiguration>

Tutorials

Format configuration for the CSV output files

Commonly the CSV format used by Trenissimo is:

• Record separator: CRLF
• Delimiter: comma (,)
• Quote: double quotes(")

If the global format needs to be changed, the Simulation project's build.xml file should be modified to execute the csvConfiguration task. Here's an example:

<target name="csv-configuration" extensionOf="init">
    <!-- formato predefinito: RFC 4180 format -->
    <trilogy:csvConfiguration formatName="RFC4180"/>
</target>

These are the default formats:

• Default:
• Excel: Excel file format (using a comma as the value delimiter). Note that the actual value delimiter used by Excel is locale dependent, it might be necessary to customize this format to accommodate to your regional settings. For example for parsing or generating a CSV file on a French system the following format will be used: <trilogy:csvConfiguration formatName="Excel" delimiter=";"/>
• InformixUnload: Default Informix CSV UNLOAD format used by the UNLOAD TO file_name operation. This is a comma-delimited format with a LF character as the line separator. Values are not quoted and special characters are escaped with \\. The default NULL string is \\\\N..
• InformixUnloadCsv: Default Informix CSV UNLOAD format used by the UNLOAD TO file_name operation (escaping is disabled.) This is a comma-delimited format with a LF character as the line separator. Values are not quoted and special characters are escaped with \\. The default NULL string is \\\\N..
• MongoDBCsv: Default MongoDB CSV format used by the mongoexport operation. This is a comma-delimited format. Values are double quoted only if needed and special characters are escaped with " A header line with field names is expected.
• MongoDBTsv: Default MongoDB TSV format used by the mongoexport operation. This is a tab-delimited format. Values are double quoted only if needed and special characters are escaped with " A header line with field names is expected.
• MySQL: Default MySQL format used by the SELECT INTO OUTFILE and LOAD DATA INFILE operations. This is a tab-delimited format with a LF character as the line separator. Values are not quoted and special characters are escaped with \\. The default NULL string is \\\\N.
• Oracle: Default Oracle format used by the SQL*Loader utility. This is a comma-delimited format with the system line separator character as the record separator.Values are double quoted when needed and special characters are escaped with " The default NULL string is "". Values are trimmed.
• PostgreSQLCsv: Default PostgreSQL CSV format used by the COPY operation. This is a comma-delimited format with a LF character as the line separator. Values are double quoted and special characters are escaped with " The default NULL string is "".
• PostgreSQLText: Default PostgreSQL text format used by the COPY operation. This is a tab-delimited format with a LF character as the line separator. Values are double quoted and special characters are escaped with " The default NULL string is \\\\N.
• RFC4180: Comma separated format as defined by RFC 4180.
• TDF: Tab-delimited format.

Also, single settings can be changed if needed, for example, to obtain an Excel format with an Italian locale:

<target name="csv-configuration" extensionOf="init">
    <trilogy:csvConfiguration formatName="Excel" delimiter=";"/>
</target>

These are the available settings:

• delimiter: the delimiter character.
• escapeCharacter: the escape character.
• nullString: the null string.
• quoteCharacter: the quote character.
• quoteMode: the quote mode. One of:
  • ALL: Quotes all fields.
  • ALL_NON_NULL: Quotes all non-null fields.
  • MINIMAL: Quotes fields that contain special characters such as field delimiters, quote characters or any of the characters in the line separator string.
  • NON_NUMERIC: Quotes all non-numeric fields.
  • NONE: Never quotes fields. When the delimiter occurs in data, the printer prefixes it with the escape character. If the escape character is not set, format validation throws an exception.

Custom output file

👷👷🏾

Examples

In-depth guide