Archive for December, 2013

The project is installed in a folder of your choosing, referred to as lapis_home. It is common to place to select /opt/fukoka/lapis as the lapis_home folder.
Inside the installation folder, we see a list of sub-folders, each with a specific purpose.

  • bin:
    the bin folder contains the command line utilities that help users and administrators interact with the lapis server.
  • conf:
    the conf folder contains your configuration files, for example the graphs (workflows) are located in this folder
  • docs:
    user guide, developer guide and administrator guide reside in this folder.
  • etc:
    the etc folder contains the configuration files relevant to the server. This would include the server properties file, the users’ XML repository, the jaas configuration file, the jaas policy file, the license file and the license signature file.
  • javadoc:
    contains the java documentation
  • lib:
    this folder contains the java archives which the server is relying on to function properly. The ext sub-folder is the location where you place your custom extensions to the server and 3rd party libraries
  • logs:
    this folder contains the log files generated by the server.
  • run:
    this folder contains files required or created when running the server. For example, the graph execution files are stored in this folder.

This sprint is almost over and I have been quite the busy guy. Below are the (important) features that have been added since my last post:

  • mail node: send emails from workflows
  • command node: execute external commands from workflows, such as perl scripts
  • encrypt/decrypt SMTP paswords
  • simplified graph execution serialisation to XML
  • password changes by users

Mail Node

I have created a core email task which connects to an smtp server using credentials stored at the server level. The server can then provide a session easily without details appearing in the workflows themselves. The password is encrypted in the property file and decrypted whenever a session is required.
Below are the connection details for my SMTP account (the password property provided is an encryption of the password):

The mail task uses XSL to transform the workflow’s XML representation as HTML email. In the next few days, I made the email task to be able to have also a text/plain part, be able to accept custom content via a custom property, reference a file list of files to attach to the email.

The mail node accepts the following parameters:

  • onSuccess: name of the arc to follow if it all goes well
  • onFailure: name of the arc to follow in case of error. Can be the same arc as the sucess arc.
  • from: email address to send the email from
  • to: email address to send the email to
  • subject: the subject of the email
  • stylesheet.html: the URL of the stylesheet to use to create the HTML alternative content of the email
  • stylesheet.text: the URL of the stylesheet to use to create the text alternative content of the email
  • xmlContent: the text of an XML document to parse. If not present, the mailer will use the graphExecution’s XML representation.
  • fileList: a path to a file containing a list of files to attach to the email. The content disposition is set to “attachment”
  • relatedFileList: a path to a file containing a list of files that can be referenced by the email. The content disposition is set to “inline”. The content id is set to a named GUID, created from the path of the image so that it can be made easier to reference from HTML emails. The cid will therefore never change for a given path.

Command node

I also added a command task, which let’s you execute system commands. With it, it is now possible to execute perl, python, LUA scripts, or any other executable you want. This is starting to make this workflow engine quite powerful.

The command node allows the workflow to execute commands that you normally would type at the prompt. This enables you top interact with the scripts or other commands that you may have already written, such as Perl scripts for example.

The command may necessitate the use of an interpreter first. For example, the perl program is not to be called directly. Instead, it must be passed as an argument to the perl interpreter: “/usr/bin/perl”.

The command node takes the following parameters:

  • onSuccess: This arc will be followed when the command completes with a return code of zero.
  • onFailure: this arc will be followed if the command cannot be instantiated or if the return code is non-zero.
  • command: the command to execute, with its arguments
  • stdout: the file to create or append to with the content of the standard output
  • stderr: the file to create or append to with the content of the error output
  • stdin: optional argument to use if the program takes its input from standard input

Below is an example of a hello world program:

<command name="perl" owner="laurent" start="true"> <!-- credit where it is due: This is simply the best hello world script of the world. -->
 <arc name="failed" to="end" /> <arc name="done" to="end" /> <parameter name="onSuccess" value="done" /> <parameter name="onFailure" value="failed" /> <parameter name="command" value="/usr/bin/perl -w"/> <parameter name="stdout" value="logs/perl.out" /> <parameter name="stderr" value="logs/perl.err" /> <parameter name="stdin"><![CDATA[package Earth;sub Greet{
         e[2])?!(push@time~~~~~~~~~~~~~~~~Zone,loc ~altime())?rotation?~~~~~~~~~~~~~q~~?The Worl ~~d?:q:[\w]::q=[\~~~~~~~~~~~~~~~~~d~a-f]=:q?..~~ ~~~?:q:.:;"42b3d3~~~~~~~~~~~~~~~~~~~~~728656c6c6f6 ~~~~~0277f627c64672~~~~~~~~~~~~~~~~~~~~~b3072796e647 ~~~~~~~42b3b3rg7d"=Ym~~~~~~~~~~~~~~~~~~~\$;~~*\;p~~~~u ~~~~~~~~~sh@_,$&;bless~~~~~~~~~~~~~~~~~~~~~~~~~$c~~~~~~~ ~~~~~~~~~o~ntine~~~~~nt~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~s=\~~~~~~~$~~~~~~~~~~~~~~~~~~~~~~~pangaea~~~~ ~~~~~~~~~~~~~~~;{l~~~~~~~~~~~~~~~~~~~~~~~~~~~~ocal@_;local$; ~~~~~~~~~~~~~~~~~="o~~~~~~~~~~~~~~~~~~~~~~~~~cean";$^A=(defi ~~~~~~~~~~~~~~~~~~~n~~~~~~~~~~~~~~~~~~~~~~~~~ed$continents)? ~~~~~~~~~~~~~~~~~~~(vec(~~~~~~~~~~~~~~~~~~~~~~$;, YYsplit(\' ~~~~~~~~~~~~~~~~~\',${\$;}~~~~~~~~~~~~~~~~~~~~~~)%3,YYsplit( ~~~~~~~~~~~~~~~~q??,$;)**2-~~~~~~~~~~~~~~~~~~~~~~(($;=Ytr/oa ~~~~~~~~~~~~~~~~eiu//)**2))=~~~~~~~~~~~~~~~~~~~~~~=28160)?q: ~~~~~~~~~~~~~~~~~.::q?!?:\'?~~~~~~~~~~~~~~~~~~~~~~\';}$^A=Ys ~~~~~~~~~~~~~~~~:\Q.\E:pack(~~~~~~~~~~~~~~~~~~~~~~\'h*\',j ~~~~~~~~~~~~~~~~~oin(q(),~~~~~~~~~~~~~~~~~~~~~~~grep{$_= ~~~~~~~~~~~~~~~~~~Ym,$,,}~~~~~~~~~~~~~~~~~~~~~~~split(" ~~~~~~~~~~~~~~~~~",@_~~~~~~~~~~~~~~~~~~~~~~~~~~[0])) ~~~~~~~~~~~~~~~~):e~~~~~~~~~~~~~~~~~~~~~~~~~~~gexe ~~~~~~~~~~~~~~~;$d~~~~~~~~~~~~~~~~~~~~~~~~~~~="s ~~~~~~~~~~~~~~ort~~~~~~~~~~~~~~~~~~~~~~~~~~<= ~~~~~~~~~~~~>,~~~~~~~~~~~~~~~~~~~~~~~~~~YY ~~~~~~~~~~~@_~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~"~~~~~~~~~~~~~~~~~~~';; s,(~|\r|\n|\s),,g;s.Y.\x7e.g; eval};Greet;'the world';]]></parameter> </command>

Other minor improvements

I have created a utility for users to change their own password or the password of another user. You must have been granted the “passwd” privilege to perform this operation (this is provided by jaas). All passwords are stored encrypted, currently in an xml document used for the user repository (I still have to write the LDAP functionality, it will come).

I fixed a small issue which workflow instances causing exception on the development environment when the server was stopped and re-started. Exceptions are now captured for each workflow to prevent exceptions being blocking to the server startup. Workflows that cause an exceptions would not be resumed.

What’s Coming Next?

I have to start thinking about bundling the release together. I’ve spent some time cleaning up the blog to look presentable and re-linked it to linked-in, google+, Twitter and Facebook. You never know who might be reading…

I also started a bit enthusiastically with the version numbering system. The next version will be version 1.2.2, to slow things down a notch. I have to look in the product backlog (yes, I do sprint planning) but I have some tidying up to do of the build system.
Maybe, I’d like to do something around queued tasks (no owner until someone changes the task ownership), Client Sessions and OAuth (to make it even more robust and ready for the Rest API I’ll be putting in for Version 2 [definitely a version number

I’ll stop blabbing on now and actually organise my sprint. If I don’t see you before next year,
Have a Happy new Year!