Introduction

This document describes all steps necessary to configure, design and deploy integration projects on Bizlynq platform. At this step user should be able to log in to the web dashboard and open designer application creating a new or opening existing project. For reference see [BizlynqAdminGuide]

Design concepts

A design project on Bizlynq Platform consists of a number of resources which may include following:

• a flow definition

• properties file

• additional configuration items

Each flow definition represents a set of steps that are followed when events are generated on a platform. Events are generated by an inbound transports that may be linked to flows. Transports are also used for some steps that require resources that are common for whole project, for example connections and supplementary services. Below is a diagram describing all major elements of fully functional integration project:

Note: Designer application is only used to configure project elements that are known at design time. Others are only referenced and can be set at deployment time. For additional information please reference [BizlynqAdminGuide]

Designer overview

Designer web application is designed in a canvas-centric approach. Major space of the screen is occupied by a canvas that visualizes state of the flow. On the left there is a smaller panel with a project tree that allows switching between process resources. It is also used as a project list for convenient switching between projects. On the right there is a pallete, used to configure flow steps.

Illustration 1

Project resources and project list pane

Resources pane contains a tree with all project resources. User can fold and unfold each part of the tree, as well as create and delete project resources.

Illustration 2

To add or remove resources user must open context menu (right click) and choose proper option. There are several options available, all of them refer to the element that was right-clicked:

Illustration 3

User can group resources using folders, create new process flows, delete and rename them.

The second role of the left pane is a project list quick switch pane. User can reach this pane by clicking ALL PROJECTS tab above the pane. Here you can see all the company projects, open or delete them or create totally new project.

Illustration 4

Palette pane

Palette pane contains a grouped list of flow elements. Those are grouped by type and presented with icons and description. Some pallete elements may be disabled, in such situation they will be greyed-out.

Illustration 5

Canvas pane

During design phase, user creates transports for the design time, creates flows, associates transports with a flow and updates flows with transformation steps linked with each other with arrows. See example canvas below:

Illustration 6

Toolbars

There are also two toolbars that supports effortless project development:

• Project tree toolbar – used for project related actions, like project validation, debugging, deployment

• Canvas toolbar – used for arrangement of flow steps, flow validation and canvas design mode switches

Flow element configuration dialog

When hovering over a flow element a spanner icon ( ) appears. This icon, when clicked, opens flow element configuration dialog. This dialog allows basic configuration and mapping of element's input data. There are also two other tabs: input and output that displays what element actually accepts as an input and what delivers as an output. In some cases these structures are editable.

Illustration 7

The mapping page by default displays graphical mapper that allows easy mapping of all elements from available data. In all modes on the left there is a tree of available data. The content of the tree depends on what actually is available. Generally output of all preceding elements is available, as well as project properties. If your data is not on the list, check if there is a path from the element providing data to the element utilizing the data. To map data link parts of the left tree with the parts of the right tree by simply dragging them.

Illustration 8

If some additional transformations are required user can enable editor view by clicking on a respective toolbar entry. Editor allows applying additional transformations based on a functions available. Auto-complete, bracket matching and function helpers are available when user is typing. Additionally Ctrl + Space can be used to trigger help dialogs when requested. All expressions and functions are part of Xpath 2.0 specification, see [XPath20Spec].

Illustration 9

On the bottom there is also an Advanced Mode switch. This brings us to the transformation text editor. This view allows to prepare a very complex and powerful transformations, provided that user understands XSL Transformations, as in [XSLT20Spec]. Advanced editor features code coloring and formatting, line numbering, auto-complete and validation.

Illustration 10

Both Input and Output tabs also supports Advanced Mode. When editable, advanced mode displays additional buttons to format, save and validate options. Code-coloring and auto-completing is also supported. Input and Output tabs in advanced mode displays and accepts code compatible with Xschema spacification as in [XSchema11Spec]

Illustration 11

Transports

Each project cannot be complete without at least two transports one inbound and one outbound. There are multiple transport types, see table Below for details.

Note: For some transports relevant services must be accessible from Bizlynq servers at least for design phase.

Most transports share these common properties, user can validate transport to ensure entered properties are valid:

Flow elements

Apart from transports directly linked to flows, generating events on Start element, user can also utilize a number of other elements to build fully working transformation logic. Flow elements have many different purposes as described below. All elements must be linked, to check if element is correctly linked and configured please validate flow (see paragraph )

Database

All Database operations require a Database transport. See paragraph 4

DATABASE
Element	Description
Database Operation	Database operation supports any dml or ddl statement that does not return any result set (insert, update, create, execute, etc.). Following configuration items are available: Property Description SQL A sql statement that will be executed. Parameters are supported if prefixed by a colon, i.e. :myParam_1. Upon change a mapping tab will be updated to allow mapping of provided params. Output optionally contains a number of affected rows and identifiers, if operation was a dml (insert, update, etc.). Some of described features are dependent on the database vendor.
Database Query	Database query works in a similar fashion as database operation above. The difference is that it returns result sets as an output and that it accemts statements that return result sets (select, procedures returning result sets, etc.). Similar properties are available: Property Description SQL A sql statement that will be executed. Parameters are supported if prefixed by a colon, i.e. :myParam_1. Upon a change a mapping tab will be updated to allow mapping of provided params. Output contains a result set returned when executing this statement. There is a hard limit of 1000 rows that can be returned

FTP

All FTP operations require a FTP transport. For supported protocols see FTP Transport definition in paragraph 4

FTP
Element	Description
FTP Get	Get operation pulls files from the server. User provides a path with optional pattern on input. Following configuration items are available: Property Description Content Processing Two options are available: Skip Body and Binary Body. First option is used when we do not want to read content, but only want to check what files are available. The other returns also a binary encoded body of file On the output element provides details of the matched files and (optional) content.
FTP Operation	FTP Operation allows some basic operations on the file server. Depending on the command below a different input is available to map. There are following properties are available: Property Description Command A type of command to be executed. Following commands are available: Rename – renames file on the server (also moves) Delete – deletes file from the server Change privileges – assigns new set of file access privileges Make directory – creates a new directory on the server There is no output available.
FTP Put	FTP Put operation uploads content to the file server. User must provide a path and a content to upload. Path must be unambiguous. There is only configuration item: Type of Content A type of content on the input. It can be either binary or text data. Input mappings depends on this setting. There is no output available.

File

File operations do not require any transport. They operate on the file system, but are limited to the fixed working directory. This operations are used mainly for caching and queuing purposes.

File
Element	Description
File Operation	File operation provides some basic functionality related to the files. Depending on the operation mapping tab requires different set of data. See properties below: Property Description Operation Following operations are supported: Delete – deletes file or directory Move – moves or renames file Copy – creates a copy of a file Operation mode Each operation may be invoked in following modes: Force – forces move and copy operation even if target exists, deletes even system files and not empty directories. Lenient – if file exists it will just ignore copy / move operation, it will complete successfully even if file to be deleted does not exists. Strict – will raise an error if target file exists or file to be deleted is missing. There is no output available.
Read File	File Read will create and store a file in the working directory. User must provide content and path where the file will be created. Path may be provided in configuration or using mapping tab, but must be unambiguous. There are following properties are available: Property Description Path A path relative to the working directory. All paths that evaluates to outside of working directory are not supported. Standard file wild-cards are also supported. On the output there is a list of files with their content.
Write File	Write File operation saves content to the file in the working directory. User must provide a path and a content to upload. Path must be unambiguous. There are following configuration items: Path A path relative to the working directory. All paths that evaluates to outside of working directory are not supported. Path must be unambiguous. File Type Either binary or text files are supported. Mapping adapts to this setting. File Access Mode There are three modes available: Create – will raise error if file exists Append – will append content to the file Overwrite – will create new file and overwrite old one if necessary There is no output available.

General

General operations enables some generic features that allows more complex mappings, error handling and others.

GENERAL
Element	Description
Call Process	Lets user call any other process, even if it is linked to the transport. Data mapped on elements input will be available on the output of start element and data mapped on the input of return element(s) within the process will be available on the output of the call process element. There is only one configuration item: Property Description Process to Call Choose a process to call, generally it is acclaimed to use processess with generic input, but provided the data format is right user can call virtually any process. Both input and output depends on the process to be called.
Catch	A catch element allows handling of errors in other elements. As a result it is not valid to draw arrows ending in Catch element. There are no configuration for this element. On the output there is an error code and a message that describes intercepted error.
Log	Log element allows writing of custom information to the same log that is available in the administration view. User provides message and code. Apart from that there is only one setting: Property Description Level Level can be one of following: Debug – will be displayed only during debugging Info – a standard entry in log Warn – a pronounced entry in log Error – a further pronounced entry in log and notification in the events pane. There is no output available.
Return	Return element allows user to end the flow at any point. Optionally output can be returned (i.e. for the call process activity). There must be no flow directed further from this element, therefore there is no output.
Sleep	Allows pausing the flow for a fixed amount of time. This element has no input/output. Property Description Sleep Time Time to wait in milliseconds

Google

Google palette contains some convenient elements that speeds development of Google oriented flows. Elements from this palette requires a REST Transport with a proper setup. See paragraph 4

GOOGLE
Element	Description
Calendar	Lets user create an event in the Google Calendar. Event title, date, duration, participants and description are all available to map. There is no other configuration necessary. There is no output in this element.
Drive	This element simplifies uploads and downloads of content to GDrive. There are three operations available: Upload – user must provide content and parent element for the upload. If the parent element will be empty, file will be created in the root folder. Download – user must provide a query for the content (see [GDriveAPIDoc]). User may optionally hint the download format he expects. If file is in different format but can be converted, then downloaded file will be a result of this conversion. List files – uses similar query as Download operation but returns only details of the files.

JMS

JMS Services may be unavailable for some plans. All elements from this palette require JMS Transport. See paragraph 4

JMS
Element	Description
Send Message	Allows user to send a message to the JMS compliant service. There are following options to be set: Property Description Request type A type of request data, following are available: XML – request data would be a part of the xml on the input JSON- xml on input will be translated to JSON Plain – input will allow to map plain data directly Response type Optional expected response type: XML – response body will be parsed and available on output JSON – json data will be translated to XML and available on output Plain – data will be presented on the output without any processing. Request method Defines whether we want to wait for the response or not: RequestResponse – wait for the reponse and present it on output Event – just send request (element output will be blank) Destination Destination where a request will be published Both input and output depends on the settings above.
Wait for Message	Wait for the message on a specific destination. Property Description Response type Optional expected response type: XML – response body will be parsed and available on output JSON – json data will be translated to XML and available on output Plain – data will be presented on the output without any processing. Destination Destination where a request will be published Output depends on the settings above.

Mail

Mail palette requires dedicated Mail Transport. See paragraph 4

MAIL
Element	Description
Mail Sender	Allows user to send message using configured mailbox. User can map all properties below or use preconfigured ones. Following properties are available: Property Description Subject A mail subject. May be overridden on mapping tab To A list of recipients separated by semicolon. May be overridden on mapping tab. Body A text body of the message. Can be overridden on mapping tab. There is no output for this flow element.

Parse

Parse activity allows to interpret various data formats, that are otherwise hard to read. Activities from this palette requires no transport.

PARSE
Element	Description
CSV Parser	Parses data in CSV (Comma Separated Values) format.User must map a binary content on input. Following properties are available: Property Description Has header If set to yes, first line will be treated as a header. Otherwise first line will be parsed as a data Row limit Maximum number of rows to parse. There is a hard limit of 1000 for the cloud deployment Separator character A field separator, usually a comma Quote character A character representing quoted fragments that will not be scanned for the occurrence of field or line separators Strict Quoting If enabled, all data outside of quoted fragments will be ignored. Output contains xml representing CSV data.
Worksheet Parser	Parses worksheet data (Currently MS Excel .xls(x) format is supported). User must provide a worksheet as a binary data on input. Following properties are available: Property Description Sheet Name A name of sheet to be parsed Start Row Number An optional row number from where to start parsing. Header Row Number This row will be marked as header on output Output will contain xml representing worksheet data.

WebService

This palette allows invoking external WebServices. All elements require either SOAP or REST Transport.

WEBSERVICE
Element	Description
REST RequestReply	Invokes REST WebService using REST Transport. On input user can map a suffix of request url and a request body. Property Description Request Method A method of this request. Currently supported are: GET POST DELETE Request type A type of request data, following are available: XML – request data would be a part of the xml on the input JSON- xml on input will be translated to JSON Plain – input will allow to map plain data directly Response type Optional expected response type: XML – response body will be parsed and available on output JSON – json data will be translated to XML and available on output Plain – data will be presented on the output without any processing. Input and output depends on the settings above.
SOAP RequestReply	Invokes SOAP WebService using SOAP Transport and predefined WSDL data. Most of the parameters are taken from WSDL configuration, therefore input and output fully depends on a transport configuration. Output will contain xml representing worksheet data.

Validating and debugging projects

During a design phase user is able to both validate and debug project. This allows to quickly test and confrm that project contains no errors and works as expected.

Validation of project and flow

Validation can be performed either on single flow or on the whole process. To validate single flow please press validate on the canvas toolbar. This will display pop-ups whenever there will be misconfiguration of flow element. Following aspects are verified:

• If links between elements are correctly

• If there are no illegal loops in the flow

• If configuration of elements is valid

• If mappings are correct (syntax and context check)

Illustration 12

Additionally user can validate whole project. To accomplish that user must click a validate project button on a project toolbar. This works in a similar fashion to the flow validation, except it will also mark other process flows with popups if it finds problems there.

Illustration 13

Debugging of projects

Bizlynq Designer also features debug option that allows to test the project during design time. For this process all transport marked as Design-time are used. To start the process user must press the test project button on the project toolbar. This will bring designer in test mode. In this mode project is read only and allows only verification of flow operation. There is an additional test phase status pane on the upper right part of the canvas. This displays a countdown timer for each test phase. If counter reaches zero test phase will end automatically. The pane also features console switch that displays console pane on the lower right part of the canvas. This console displays data from application log plus data marked as debug. During test phase user may need to trigger proper transport manually (i.e. by sending a mail to the monitored mailbox). As processing advances gray arrows are pained black and small bugs icons appears near each traversed element. If the icon is green, then processing advanced without error, if the icon is red, there was an error.

Illustration 14

Either way user can click the bug icon and open dialog with live input and output data for each activity.

Illustration 15

When finished, user can either click “Turn off test” on a test phase status pane or click again test project button on a toolbar.

Versioning and deployment

Bizlynq features two deployment methods and a simple version control system that automatically creates snapshots of project when deployment is requested.

Quickploy

If proper user configuration is set (see [BizlynqAdminGuide] user settings), user can quickly deploy his project to the designated environment. During deployment phase, Bizlynq matches new transports to the project and deploys them together with a single click.

Deployment workflow

Additionally user can request a deployment to other environments. This begins a workflow which may end in project being deployed on other machines (or being rejected) more on that in the [BizlynqAdminGuide] section deployment.

Illustration 16

Versioning

Version control logic is located on the footer pane of the Designer. There is a label presenting information related to the currently active version, each version can be in one of 4 states:

• Draft – the only editable statement

• Requested deployment – read only state after workflow is initiated.

• Deployed to Test and Deployed to Prod – when project version is deployed

This label when clicked allows switching between versions, displaying a list of versions, their status and last update date.

Illustration 17

If user deployed a version and wants to continue working on a project, then a new version from the one deployed must be created. This can be accomplished by Create New Version button on project toolbar.

Illustration 18

Dictionary

Additional lecture

References

BizlynqAdminGuide: Bizlynq Systems, Bizlynq Administration and Management Guide, v1.0

XPath20Spec: W3C, XML Path Language (XPath) 2.0 (Second Edition), 3 January 2011

XSLT20Spec: W3C, XSL Transformations (XSLT) Version 2.0, 23 January 2007

XSchema11Spec: W3C, XML Schema Definition Language (XSD) 1.1, 5 April 2012

Jms11Specification: Oracle, Java(TM) Message Service Specification , Final Release 1.1

rfc6749: IEFT, The OAuth 2.0 Authorization Framework, October 2012

crontabDocumentation: The IEEE and The Open Group, The Open Group Base Specifications Issue 7 - crontab, 2013 Edition

WS-Security: OASIS Open, WSS: SOAP Message Security, WS-Security 2004

SOAP12: W3C, SOAP Version 1.2 Part 1: Messaging Framework , 27 April 2007, Second Edition

GDriveAPIDoc: Google Inc., Drive API Reference, 2014