Introduction
The integration of Smallworld with external business processes is a common pursuit for many organizations that use the platform. Realworld has two products which can serve as foundations for such integration solutions in very different ways – SmallWorld Enterprise Gateway (SWEG) and Data Transit System (DTS). This article aims to show how the two compare from the points of view pertinent to this perspective and to show when either or both should be chosen.
Data Proximity
The main consideration when integrating Smallworld with external systems is what kind of data proximity is required by the processes.
When a process needs to make very large numbers of low latency calls into a data source, it is beneficial to bring that data source as close to the process as possible, so a mirror of the data in a system that the process can work with may be the best solution. This is what SWEG does. It creates and maintains a faithful mirror of Smallworld in Oracle and can keep it synchronized both ways.
Conversely, in most situations where a data mirror is not required, the best solution is to provide live access directly into and out of Smallworld. This is where DTS comes in – with DTS, anything you can query or execute in Smallworld can be seen from outside through multiple APIs and anything you can query or execute outside of Smallworld can be seen from Magik as native.
Oracle Modelling or Generic Translating
SWEG is built to model Smallworld in Oracle. This means that all Smallworld DS constructs are rebuilt using Oracle objects: joins (including hetero) become constraints and intermediate tables, SW geometries become SDO geometries, manifolds become networks, keys become PKs, indexes are mirrored, etc. This is extremely beneficial for processes built to use these Oracle objects.
DTS also provides access to any Smallworld DS constructs (as well as any definable Magik exemplars), but it takes a wholly different approach. With DTS, what you get when you make requests into Smallworld is the best possible representation in the context from which you make the request. If it’s a REST call, you get a JSON, if it’s Java you get a Java object, etc. Geometries will become GeoJSONs, WKTs, WKBs, Java or .NET objects, depending on who’s asking, join field values become lists of objects and any Magik objects are read as identically structured objects in the target environment.
Faithful Data Warehousing or Cloud Microservices
This is the main conceptual difference between SWEG and DTS. While in essence they both have the capability to get things in and out of Smallworld, SWEG does it by bulk transfer and translation to and from a faithful Oracle representation, while DTS fully liberalizes live access to and from a large array of paradigms.
Another way to view this is with SWEG as a highly specialized pipeline between 2 systems (Smallworld and Oracle) and DTS as a graph with one or more nodes in Smallworld.
Performance and Scaling
Both products are optimized for performance, but in very different ways. SWEG is optimized for throughput – getting as much data from one side to the other as quickly as possible. DTS, on the other hand, is optimized for fast routing of a large numbers of concurrent requests.
Scaling is an integral part of both products’ architecture. SWEG accomplishes it by opening and leveraging multiple Smallworld sessions on one or more machines, meanwhile DTS takes a containerized approach and spins up containers to run multiple sessions of whichever type is needed. Both can do it automatically provide plenty of options for configuration.
Security
The different approaches of the two products mean that their security solutions are also quite different.
SWEG is designed to be used inside an organization’s network, with no real attack avenues and no publicly exposed endpoints. As such, its only security concern is its connection to Oracle, which could be routed through public networks, and where it is fully capable of leveraging Oracle’s SSL paradigm.
DTS, however, is built from the ground up with security in mind. Its purpose is to link together all kinds of systems and even provide web APIs, so every part of it is secured. Communication with all data sources is encapsulated and secured using each data source’s solution, internal communication between DTS modules uses x509 certificates for authorization and authentication and rotated AES key, specific for each communication channel. In addition, DTS’s web endpoints are generated with full documentation (WSDL, WADL, OpenAPI) so that they can effortlessly be secured using a reverse proxy solution.
Other Considerations
- Incremental Updates or Live Access: Do the business processes work best when they are fed or triggered by incremental updates, or would it be better if they always had live access to the current version of the data?
- Get Smallworld to do the work: It can be transformative to offload all sorts of computations into Smallworld and just ask for results by invoking routines directly. With DTS, this is easily done.
- A Look Outside: Integrating web APIs and external data sources into Smallworld operations is easy and direct with DTS.
Making a Choice by Example
A common example of Smallworld integration is with an ERP system. We can use this as a case study to see which product would work best.
SWEG is a good choice if the ERP system:
- Is Oracle-based.
- Benefits from storing deep asset information in proximity (i.e. processes large amounts of asset data consistently).
- Does not require data to be up to date in real time.
DTS is a good choice if the ERP system:
- Integrates with web APIs or includes some type of programmability.
- Benefits from receiving tailored responses, rather than processing raw data.
- Benefits from real time updates.
- Benefits from integrating with other systems as well.
Complementarity
There are also cases where using both products yields the best results. An example of this would be integration between Smallworld and a SCADA system that is Oracle-based.
Many SCADA systems benefit from having a mirror of asset and network data in proximity and can work with Oracle (e.g. ADMS). As such, SWEG fits in well. But SCADA also generates plenty of real-time events and pieces of data that need to be acted upon back in Smallworld, which is where DTS with its integration of web APIs, event engines (Kafka) and routine execution capabilities can bridge the gap.
Simplified Conclusion
Do you want to mirror Smallworld in Oracle as faithfully and efficiently as possible and keep it synchronized? SWEG is the answer.
For any other integration you can think of, there is DTS.
If you are still confused about what DTS and SWEG can do, let’s have a talk, and we can unravel the mysteries of each of them.
