======== Tutorial ======== ------------ Requirements ------------ Applications ============ The PSCAD Automation Library requires the following tools are installed: * PSCAD 4.6.3 * Python 3.x (such as Python 3.7.1) * PIP - Python's package manager (included with Python 3.4 and later) * PyWin32 - Python extensions for Microsoft Windows Installation of the above programs is beyond the scope of this tutorial. Automation Library ================== You can check which version of the Automation Library is installed by executing:: py -m mhrc.automation If multiple versions of Python have been installed, each installation will need its own copy of the Automation Library, so you may want specify the Python version being queried:: py -3.7 -m mhrc.automation PSCAD's "Update Client" will automatically install the Automation Library into the latest Python 3.x installation. If you wish to add the Automation Library to a different Python 3.x installation, you would execute a ``PIP install`` command similar to the following from the ``C:\Program Files (x86)\PSCAD\AutomationLibrary\463\Installs\PyAL`` directory: .. parsed-literal:: py -3.4 -m pip install mhrc_automation-|release|-py3-none-any.whl -------------------------- My First Automation Script -------------------------- Running the Tutorial ==================== The following :download:`script ` is the "simplest" PSCAD automation script that does something useful. It: * launches PSCAD, * loads the "Tutorial" workspace, * runs all simulation sets in the "Tutorial" workspace, and * quits PSCAD. To keep this first example simple, no error checking of any kind is done. Also, no feedback from the simulation is retrieved for the same reason. .. literalinclude:: step-00.py Many assumptions are being made here. #. Certificate licensing is used, and a license certificate can be automatically acquired. Ensure "Retain certificate" is selected under "Certificate Licensing ➭ Termination Behaviour" and ensure you hold a valid certificate when you exit PSCAD before running the above script. #. The "Public Documents" directory is located at `C:\\Users\\Public\\Documents`. #. The 64-bit version PSCAD 4.6.3 is installed. Finding the Tutorial ==================== We can relax the last two restrictions by retrieving the correct location of the "Public Documents" directory from PyWin, and searching inside that directory for the "Tutorial" workspace. Doing so will allow the scripts in this tutorial to be downloaded and executed without modification by the reader. Alter the `import` statements at the top of the script to read: .. literalinclude:: step-01.py :lines: 2-3 To retrieve the "Public Documents" folder, add the following statement below the imports: .. literalinclude:: step-01.py :start-at: Public Documents :end-at: shell.SHGetFolderPath Beneath this statement, add another which will search inside the "PSCAD" folder for a subdirectory which contains the desired `Tutorial.pswx` file: .. literalinclude:: step-01.py :start-at: Find the "Tutorial" workspace :end-at: if "Tutorial.pswx" in files Finally, change the `pscad.load()` statement to load the `Tutorial.pscx` workspace from that directory: .. literalinclude:: step-01.py :start-at: Load the tutorial workspace :end-at: os.path.join(tutorial_dir .. only:: builder_html See :download:`the resulting script `. Logging ======= Add ``logging`` to the current imports, and add the ``logging.basicConfig`` statement below: .. literalinclude:: step-02.py :start-at: import :end-at: format If you now run the :download:`script `, you should see something similar to following output. The paths, port numbers, and process ID's may be different: .. literalinclude:: step-02.out In the next steps, we’ll be adding additional logging to our script. The logging already built into the automation library can be squelched to warnings and above, so it is easier to see our own log messages. After the ``logging.basicConfig( … )`` line, add the following: .. literalinclude:: step-03.py :start-at: Ignore INFO msgs :end-at: getLogger('main') Next, add logging lines to print out the location of the "Public Documents" and "tutorial" directories:: LOG.info("Public Documents : %s", public_dir) LOG.info("Tutorial directory: %s", tutorial_dir) If you now run the :download:`script `, you should see something similar to following output. Again, the paths may be different, which is of course the whole point of this exercise: .. literalinclude:: step-03.out Error Handling ============== If the automation controller cannot launch PSCAD, it will log the reason why to the console and return the value ``None``. Other problems may be signalled by an “Exception” being raised. A well behaved script must catch these exceptions, and properly clean up. This cleanup should include closing PSCAD. Replace the final 3 commands in the :download:`script ` with the following: .. literalinclude:: step-04.py :start-at: if pscad :end-at: Failed to launch PSCAD Whether or not an exception occurs during the loading of the workspace or running all simulation sets, due to the ``try: ... finally: ...`` block, PSCAD will always be terminated before the script terminates. Launch Options ============== Minimize PSCAD -------------- When PSCAD is started, it normally opens the PSCAD window. During automation, it is sometimes desirable to launch PSCAD with its window “minimized”, reducing the amount “screen flickering”, as well as the chance a stray mouse click could “poison” a running automation. Add ``minimized=True`` to the ``launch_pscad()`` call: .. literalinclude:: step-05.py :start-at: Launch PSCAD :end-at: minimize=True Run this new :download:`script `. You should still see log messages in the Python Shell, and PSCAD appear in the Task Bar, but no PSCAD window should open. .. note:: PSCAD remembers where it is on the screen, including whether or not it is “maximized”. When it is launched, it restores itself to that position and, if required, maximized state. The automation library cannot override the remembered “maximized” state. If PSCAD remembers that it was last “maximized”, it will maximize itself, regardless of whether it was launched in a “minimized” state by the automation library. Manually launching PSCAD, de-maximizing it, and quitting it will fix this issue. PSCAD Version ------------- The :meth:`~mhrc.automation.launch_pscad()` method does a bit of work behind the scenes. If more than one version of PSCAD is installed, it tries to pick the best version to run. "Best" in this context means: * not an "Alpha" version, if other choices exist, then * not a "Beta" version, if other choices exist, then * not a 32-bit version, if other choices exist, finally * the "lexically largest" version of the choices that remain. Instead of letting :meth:`.launch_pscad` choose the version, the script can specify the exact version to use with the ``pscad_version=...`` parameter. For example, to launch the 64-bit version of PSCAD 4.6.3, use:: pscad = mhrc.automation.launch_pscad(pscad_version='PSCAD 4.6.3 (x64)') A controller object may be queried to determine which versions of PSCAD are installed. Replacing the ``launch_pscad()`` statement with the following will mimic its selection process: .. literalinclude:: step-06.py :start-at: mhrc.automation.controller() :end-at: launch_pscad With the above changes to the :download:`script `, the output may be: .. literalinclude:: step-06.out Fortran Version --------------- In a similar fashion, the controller object may be queried for which versions of FORTRAN are installed. Before the ``launch_pscad()`` line, add the following: .. literalinclude:: step-06a.py :start-at: Get all installed FORTRAN compiler versions :end-at: Selected FORTRAN version Add the ``, fortran_version=fortran`` parameter to the ``launch_pscad()`` call: .. literalinclude:: step-06a.py :start-at: Launch PSCAD :end-at: fortran_version=fortran With the above changes to the :download:`script `, the output may be: .. literalinclude:: step-06a.out Matlab Version -------------- In a similar fashion, the controller object may be queried for which versions of Matlab are installed. Matlab, unlike FORTRAN, is not required for PSCAD to run, so there may be no versions of Matlab installed. Before the ``launch_pscad()`` line, add the following: .. literalinclude:: step-06b.py :start-at: Get all installed Matlab versions :end-at: Selected Matlab version Add the ``, matlab_version=matlab`` parameter to the ``launch_pscad()`` call: .. literalinclude:: step-06b.py :start-at: Launch PSCAD :end-at: matlab_version=matlab With the above changes to the :download:`script `, the output may be: .. literalinclude:: step-06b.out PSCAD Settings ============== To ensure reliable, repeatable tests, the automation library instructs PSCAD to use the default settings, ignoring any stored user settings. The :meth:`pscad.settings() ` method must be used to change any application settings to the desired value. .. .. include:: .. .. |vertical-ellipsis| unicode:: U+022EE .. VERTICAL ELLIPSIS The following :download:`script ` may be used to determine what settings exist, and their current values: .. literalinclude:: settings.py And produces output similar to: .. literalinclude:: settings.out :end-at: backup_freq :append: ⋮ ⋮ ⋮ ⋮ To change settings from their defaults, pass one or more ``key=value`` arguments to the :meth:`pscad.settings() ` method:: pscad.settings(MaxConcurrentSim=8, LCP_MaxConcurrentExec=8) .. note:: All settings are automatically converted to strings before passing to PSCAD. There is no functional difference between ``pscad.settings(MaxConcurrentSim=8)`` and ``pscad.settings(MaxConcurrentSim="8")``. PSCAD returns all settings as strings. If a numeric setting is expected, the Python script is responsible for converting the string into an integer. Booleans may be returned as ``"true"`` or ``"false"``, without the first letter capitalized; again the Python script must accept responsibility for converting the strings into Python boolean values ``True`` and ``False`` if required. Projects ======== To run all loaded projects, one at a time, we first obtain a list of all projects. From this list, we filter out any libraries, which are not runnable. Then, we retrieve a :class:`~.Project` controller for each project, and call :meth:`Project.run() <.Project.run>` on each one, in turn. Replace the ``pscad.run_all_simulation_sets()`` line with the following: .. literalinclude:: step-07.py :start-at: Get a list of all projects :end-at: Run '%s' complete This :download:`script ` would produce: .. literalinclude:: step-07.out Simulation Sets =============== When a workspace contains multiple projects, they may be required to run together or in a particular sequence. A simulation set is often used to control the collection of projects. Instead of blindly running all projects in the workspace, or simply all simulation sets, we may retrieve the list of simulations sets and run each simulation set individually, under the control of our script. If no simulation sets are found, we can fall back to running each project separately. Replace the code we just added, with this code instead: .. literalinclude:: step-08.py :start-at: pscad.workspace() :end-at: Run '%s' complete This version of the :download:`script ` runs all projects simultaneously, producing the following output: .. literalinclude:: step-08.out If, instead of loading ``Tutorial.pswx``, we just loaded ``vdiv.pscx`` project: .. literalinclude:: step-09.py :start-at: # Load only the 'voltage divider' project :end-at: "vdiv.pscx" there are no simulation sets, so the ``else:`` path is taken in our :download:`script `, and the ``vdiv`` project is run: .. literalinclude:: step-09.out ---------------- Event Monitoring ---------------- Event Handlers ============== The Automation Library will allow you to observe the communications between the automation library and the PSCAD process. At the top of the file, add the following “Handler” class: .. literalinclude:: step-10.py :start-at: ElementTree :end-at: pass :lines: 1,7- Following the successful launch of PSCAD, we can create an instance of our ``Handler`` class, and attach it to the PSCAD instance. After the ``if pscad:``, add the following two lines: .. literalinclude:: step-10.py :start-at: if pscad :end-at: add_handler After downgrading some of the earlier ``LOG.info(...)`` messages to ``LOG.debug(...)`` message, if we run this :download:`script `, we might see: .. literalinclude:: step-10.out As messages are received from PSCAD, the automation library sends the message to each registered handler, by calling ``handler.send(msg)``. The handler can do pretty much whatever it wants with the message. If it consumes the message, and doesn’t want any subsequent handler from seeing it, the handler should return ``True``. After a period of time when no messages have been sent back from PSCAD, the automation library also sends a blank message to the handler, by calling ``handler.send(None)``. This allows the handler a chance to do additional processing. For instance, the Automated Test Suite’s ``ProgressHandler`` uses those opportunities to send a ``get-run-status`` command to the various projects, in order to track ``% complete``. Removing Handlers ================= If the script decides a certain handler is no longer required, the script can remove it by calling ``pscad.remove_handler( handler )``. Automatic Removal ================= Since the handler is receiving messages from PSCAD, it is usually in the best position to determine when a particular process is complete. The handler can indicate this to the automation library, by returning the special value ``StopIteration``, or by raising the ``StopIteration`` exception. When this happens, the automation library will automatically remove the handler. For example, the ``Load`` process is complete when a load event is found with a ``file-type`` of ``files`` and a ``status`` of ``END``. Change the Handler’s ``send()`` method to the following code: .. literalinclude:: step-12.py :pyobject: Handler.send When this new :download:`script ` is run, this much shorter output results: .. literalinclude:: step-12.out Build Events ============ Build Events are generated when PSCAD builds and runs cases. A build event handler might be installed just before executing the run command, and removed just after the run command finished. Code to do thing might look like this:: handler = BuildEventHandler() pscad.add_handler(handler) project.run() pscad.remove_handler(handler) This is a lot of boiler-plate code. As a convenience, the automation library will perform the ``add_handler`` and ``remove_handler`` calls itself, if the handler is passed to the ``run( )`` command directly:: project.run( BuildEventHandler() ) .. note:: When used this way, it is the handler’s auto-termination that indicates the ``run( )`` command is complete. The handler must properly detect the completion of the run task. Add an import for ``mhrc.automation.handler``: .. literalinclude:: step-13.py :start-at: import :end-at: ElementTree Replace the ``Handler`` code from the previous section with the following: .. literalinclude:: step-13.py :pyobject: BuildEventHandler Here, we are extending a standard ``BuildEvent`` handler. Its ``send()`` method already looks for build event messages, and forwards them to the ``_build_event()`` method. Its ``_build_event()`` method looks for matching ``BEGIN`` and ``END`` messages, and returns ``StopIteration`` when the final ``END`` message is found. This standard ``BuildEvent`` handler in turn extends an ``AbstractHandler``, which implements the required do-nothing ``close()`` method. As such, most of the required work has already been done for us. We just need to override ``_build_event()``, and add ``return super()._build_event(msg, event)`` at the end. Remove the previous ``pscad.add_handler( )`` call, and replace the ``project.run()`` call with: .. literalinclude:: step-13.py :start-at: project.run :end-at: project.run When this :download:`script ` is run, the following output is produced: .. literalinclude:: step-13.out Our handler is just displaying the events as they occur. It could do more interesting things. For instance, when it receives a ``BEGIN`` message, it could record the ``elapsed`` time; when it receives a matching ``END`` message, it could subtract the ``elapsed`` time from the time it recorded earlier, giving the time required for each task: .. literalinclude:: step-14.py :pyobject: BuildEventHandler Running this revised :download:`script ` script would produce: .. literalinclude:: step-14.out The handler can store information collected during the run, that may be accessed afterwards. Of course, to do so, you would need to hold onto a reference to the handler:: handler = BuildEventHandler() project.run( handler ) info = handler.get_interesting_data() -------------------- Build & Run Messages -------------------- Build Messages ============== When PSCAD builds a project, the build messages are recorded. The script can retrieve these build messages as an XML document. After the ``project.run(…)`` line, add the following: .. literalinclude:: step-20.py :start-at: project.run :end-at: msg.status This :download:`script ` would produce: .. literalinclude:: step-20.out :prepend: ⋮ ⋮ ⋮ ⋮ :start-at: EMTDC RUN Here, we are just extracting the scope, status, and text of the messages. Other fields, such as label and component references could also be extracted. Run Messages ============ After EMTDC has run the project, the run messages may also be retrieved. Unlike the build messages, the run messages are returned as an unstructured blob of text. Immediately after the above code, add the following lines: .. literalinclude:: step-21.py :start-at: "-"*60 :end-at: print(output) When run, this change to the :download:`script ` adds the run messages to the output: .. literalinclude:: step-21.out :prepend: ⋮ ⋮ ⋮ ⋮ :start-after: Creating EMTDC executable