eXtremeDB for HPC

eXtremeDB for High Performance Computing was built to handle the scalability and analytics needs of financial and IoT Big Data systems. It offers “in-database” pipelined vector-based analytics, unique features for time series support, LUA stored procedure language, and a distributed query engine that enables parallel query execution across shards for linear scalability.

What makes eXtremeDB for HPC different?

Many database systems include one or two of the same performance-enhancing features such as in-memory data storage, columnar data handling, fast native APIs as well as high performance SQL, and a short execution path. But you’d be hard-pressed to find all of them in one database system, and solutions offering some of these features may also lack “real” database capabilities such as ACID transactions. In contrast, eXtremeDB offers all of them plus a rich assortment of additional development and run-time features.

Pipelining and other features to optimize time-series data

Pipelining is the programming technique in eXtremeDB that accelerates processing by combining the database system’s vector-based statistical functions into assembly lines of processing for market or time-series data, with the output of one function becoming input for the next. Calculations are pipelined in order to keep data within CPU cache during its transformation by multiple functions. Without pipelining, interim results from each function would be transferred back and forth between CPU cache and main memory, imposing significant latency due to the relatively lower-bandwidth front side bus (FSB) or quick path interconnect (QPI) between the two.

Elastic scalability via sharding

Configure sharding easily with xPanel for database administration/monitoring. Sharding horizontally partitions a logical database into multiple physical databases that can be distributed on any number of virtual or physical servers to take advantage of multiple I/O channels and multiple CPUs/CPU cores.

Distributed query processing

When a database is partitioned (sharded), a client application’s query can be distributed to all shards to be executed in parallel by each shard’s server. The eXtremeDB Distributed Query Engine handles this on behalf of, and transparently to, the client application and collects each shard server’s partial result set, merges them, and presents the complete result set.

Time-series processing

eXtremeDB supports database designs that optimize time-series processing. Over 150 built-in vector based statistical functions can be pipelined for the lowest possible latency. Combine row-based and column-based layouts, in order to best leverage the CPU cache speed. The internal transaction and memory managers provide maximum efficiency working with multi-threaded applications on multi-processor systems.

LUA/Python/C/C++ stored procedures (UDF)

Lua is a very elegant, popular and easy-to-adopt scripting language. Lua’s sophisticated and blazingly fast dynamic Just-In-Time compiler (LuaJIT) and small footprint makes it a great procedural language to develop complex database user-defined functions and stored procedures for the eXtremeDB SQL server.