Lapis server

Posted: 31 March 2014 in Teamsite

Hello all,

in this version of the Lapis server, we have made sure that workflows (or graph executions) can be interrupted at any time and resumed afterwards via the API, command line or the web application.

The web application now contains a link in the workflows menu to that effect and also a warning when a workflow is not currently alive.
A workflow which is not alive can be resumed by from the API, the web application and a command line tool.

We have also created the default configuration for the logging API, which now creates and rotates files in the logs directory of the server.

We have also added the capability to add a custom instantiation form to each graph to make the whole process a lot more user friendly.
We have integrated our own JavaScript form cloning solution to make adding and removing elements on the fly a breeze

As always, work carries on…

Lapis Server enhancements

Posted: 19 March 2014 in Teamsite

Hello All,

Recently, there has been a fair amount of work going on Lapis Server.

The lapis server email task now supports multiple addresses for to, cc and bcc fields.
The audit trail is now complete. It is possible to track who created, activated, rejected or completed a graph execution or a token.
The javadocs are now available in jar form so that they are easier to incorporate into the development environment of developers. They are also available through the web application.
Talking of which, the web application now is taking (a nice) shape but there is still plenty to do. It is a big API, and I’d like to have most of it represented in the web app, so I’ll crack on with it this month.

Here’s a screenshot of the webapp’ve view of a graph execution (a.k.a. a workflow). You can see it has a list of active tokens (a.k.a. tasks) that a user can complete (or that a program is currently completing), a list of previously completed and rejected tokens, the token attributes and properties.

2013 in review

Posted: 5 January 2014 in Uncategorized

The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.

Here’s an excerpt:

The concert hall at the Sydney Opera House holds 2,700 people. This blog was viewed about 13,000 times in 2013. If it were a concert at Sydney Opera House, it would take about 5 sold-out performances for that many people to see it.

Click here to see the complete report.

The project is installed in a folder of your choosing, referred to as lapis_home. It is common to place to select /opt/fukoka/lapis as the lapis_home folder.
Inside the installation folder, we see a list of sub-folders, each with a specific purpose.

  • bin:
    the bin folder contains the command line utilities that help users and administrators interact with the lapis server.
  • conf:
    the conf folder contains your configuration files, for example the graphs (workflows) are located in this folder
  • docs:
    user guide, developer guide and administrator guide reside in this folder.
  • etc:
    the etc folder contains the configuration files relevant to the server. This would include the server properties file, the users’ XML repository, the jaas configuration file, the jaas policy file, the license file and the license signature file.
  • javadoc:
    contains the java documentation
  • lib:
    this folder contains the java archives which the server is relying on to function properly. The ext sub-folder is the location where you place your custom extensions to the server and 3rd party libraries
  • logs:
    this folder contains the log files generated by the server.
  • run:
    this folder contains files required or created when running the server. For example, the graph execution files are stored in this folder.

This sprint is almost over and I have been quite the busy guy. Below are the (important) features that have been added since my last post:

  • mail node: send emails from workflows
  • command node: execute external commands from workflows, such as perl scripts
  • encrypt/decrypt SMTP paswords
  • simplified graph execution serialisation to XML
  • password changes by users

Mail Node

I have created a core email task which connects to an smtp server using credentials stored at the server level. The server can then provide a session easily without details appearing in the workflows themselves. The password is encrypted in the property file and decrypted whenever a session is required.
Below are the connection details for my SMTP account (the password property provided is an encryption of the password):

mail.smtp.host=smtp.gmail.com
mail.smtp.port=465
mail.smtp.starttls.enable=true
mail.smtp.ssl.enable=true
mail.smtp.auth.enable=true
mail.smtp.starttls.enable=true
mail.smtp.auth.username=lpicquet
mail.smtp.auth.password=DTeF3oscP65GRg8sL9KiIVb+KIgohbQ3t

The mail task uses XSL to transform the workflow’s XML representation as HTML email. In the next few days, I made the email task to be able to have also a text/plain part, be able to accept custom content via a custom property, reference a file list of files to attach to the email.

The mail node accepts the following parameters:

  • onSuccess: name of the arc to follow if it all goes well
  • onFailure: name of the arc to follow in case of error. Can be the same arc as the sucess arc.
  • from: email address to send the email from
  • to: email address to send the email to
  • subject: the subject of the email
  • stylesheet.html: the URL of the stylesheet to use to create the HTML alternative content of the email
  • stylesheet.text: the URL of the stylesheet to use to create the text alternative content of the email
  • xmlContent: the text of an XML document to parse. If not present, the mailer will use the graphExecution’s XML representation.
  • fileList: a path to a file containing a list of files to attach to the email. The content disposition is set to “attachment”
  • relatedFileList: a path to a file containing a list of files that can be referenced by the email. The content disposition is set to “inline”. The content id is set to a named GUID, created from the path of the image so that it can be made easier to reference from HTML emails. The cid will therefore never change for a given path.

Command node

I also added a command task, which let’s you execute system commands. With it, it is now possible to execute perl, python, LUA scripts, or any other executable you want. This is starting to make this workflow engine quite powerful.

The command node allows the workflow to execute commands that you normally would type at the prompt. This enables you top interact with the scripts or other commands that you may have already written, such as Perl scripts for example.

The command may necessitate the use of an interpreter first. For example, the perl program helloWorld.pl is not to be called directly. Instead, it must be passed as an argument to the perl interpreter: “/usr/bin/perl helloWorld.pl”.

The command node takes the following parameters:

  • onSuccess: This arc will be followed when the command completes with a return code of zero.
  • onFailure: this arc will be followed if the command cannot be instantiated or if the return code is non-zero.
  • command: the command to execute, with its arguments
  • stdout: the file to create or append to with the content of the standard output
  • stderr: the file to create or append to with the content of the error output
  • stdin: optional argument to use if the program takes its input from standard input

Below is an example of a hello world program:

<command name="perl" owner="laurent" start="true"> <!-- credit where it is due: This is simply the best hello world script of the world. http://www.perlmonks.org/?node_id=329174 -->
 <arc name="failed" to="end" /> <arc name="done" to="end" /> <parameter name="onSuccess" value="done" /> <parameter name="onFailure" value="failed" /> <parameter name="command" value="/usr/bin/perl -w"/> <parameter name="stdout" value="logs/perl.out" /> <parameter name="stderr" value="logs/perl.err" /> <parameter name="stdin"><![CDATA[package Earth;sub Greet{
                 %_=('Y','~');$_='$;=!(Middle
              Earth.age~~~<Eart~~~~~~~~~~~~~h
           .age)?!(defined$ti~~~~~~~~~~~mez~~~On
         e[2])?!(push@time~~~~~~~~~~~~~~~~Zone,loc ~altime())?rotation?~~~~~~~~~~~~~q~~?The Worl ~~d?:q:[\w]::q=[\~~~~~~~~~~~~~~~~~d~a-f]=:q?..~~ ~~~?:q:.:;"42b3d3~~~~~~~~~~~~~~~~~~~~~728656c6c6f6 ~~~~~0277f627c64672~~~~~~~~~~~~~~~~~~~~~b3072796e647 ~~~~~~~42b3b3rg7d"=Ym~~~~~~~~~~~~~~~~~~~\$;~~*\;p~~~~u ~~~~~~~~~sh@_,$&;bless~~~~~~~~~~~~~~~~~~~~~~~~~$c~~~~~~~ ~~~~~~~~~o~ntine~~~~~nt~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~s=\~~~~~~~$~~~~~~~~~~~~~~~~~~~~~~~pangaea~~~~ ~~~~~~~~~~~~~~~;{l~~~~~~~~~~~~~~~~~~~~~~~~~~~~ocal@_;local$; ~~~~~~~~~~~~~~~~~="o~~~~~~~~~~~~~~~~~~~~~~~~~cean";$^A=(defi ~~~~~~~~~~~~~~~~~~~n~~~~~~~~~~~~~~~~~~~~~~~~~ed$continents)? ~~~~~~~~~~~~~~~~~~~(vec(~~~~~~~~~~~~~~~~~~~~~~$;, YYsplit(\' ~~~~~~~~~~~~~~~~~\',${\$;}~~~~~~~~~~~~~~~~~~~~~~)%3,YYsplit( ~~~~~~~~~~~~~~~~q??,$;)**2-~~~~~~~~~~~~~~~~~~~~~~(($;=Ytr/oa ~~~~~~~~~~~~~~~~eiu//)**2))=~~~~~~~~~~~~~~~~~~~~~~=28160)?q: ~~~~~~~~~~~~~~~~~.::q?!?:\'?~~~~~~~~~~~~~~~~~~~~~~\';}$^A=Ys ~~~~~~~~~~~~~~~~:\Q.\E:pack(~~~~~~~~~~~~~~~~~~~~~~\'h*\',j ~~~~~~~~~~~~~~~~~oin(q(),~~~~~~~~~~~~~~~~~~~~~~~grep{$_= ~~~~~~~~~~~~~~~~~~Ym,$,,}~~~~~~~~~~~~~~~~~~~~~~~split(" ~~~~~~~~~~~~~~~~~",@_~~~~~~~~~~~~~~~~~~~~~~~~~~[0])) ~~~~~~~~~~~~~~~~):e~~~~~~~~~~~~~~~~~~~~~~~~~~~gexe ~~~~~~~~~~~~~~~;$d~~~~~~~~~~~~~~~~~~~~~~~~~~~="s ~~~~~~~~~~~~~~ort~~~~~~~~~~~~~~~~~~~~~~~~~~<= ~~~~~~~~~~~~>,~~~~~~~~~~~~~~~~~~~~~~~~~~YY ~~~~~~~~~~~@_~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~"~~~~~~~~~~~~~~~~~~~';; s,(~|\r|\n|\s),,g;s.Y.\x7e.g; eval};Greet;'the world';]]></parameter> </command>

Other minor improvements

I have created a utility for users to change their own password or the password of another user. You must have been granted the “passwd” privilege to perform this operation (this is provided by jaas). All passwords are stored encrypted, currently in an xml document used for the user repository (I still have to write the LDAP functionality, it will come).

I fixed a small issue which workflow instances causing exception on the development environment when the server was stopped and re-started. Exceptions are now captured for each workflow to prevent exceptions being blocking to the server startup. Workflows that cause an exceptions would not be resumed.

What’s Coming Next?

I have to start thinking about bundling the release together. I’ve spent some time cleaning up the blog to look presentable and re-linked it to linked-in, google+, Twitter and Facebook. You never know who might be reading…

I also started a bit enthusiastically with the version numbering system. The next version will be version 1.2.2, to slow things down a notch. I have to look in the product backlog (yes, I do sprint planning) but I have some tidying up to do of the build system.
Maybe, I’d like to do something around queued tasks (no owner until someone changes the task ownership), Client Sessions and OAuth (to make it even more robust and ready for the Rest API I’ll be putting in for Version 2 [definitely a version number
changer!])

I’ll stop blabbing on now and actually organise my sprint. If I don’t see you before next year,
Have a Happy new Year!