Glayer is a GPU accelerated database engine focused on fast data aggregation, filtering and visualisation. GLayer is capable of performing analytical queries on extremely large datasets in a scope of milliseconds with special emphasis on spatial data, spatial aggregation and interactivity.
Basic concept
Generally GPU uses thousands of lightweight processing cores, leverages data parallelism, and has high memory throughput. This architecture is known to be very effective for computer graphics. In the GLayer concept, the dataset is splitted into smaller parts and each part is processed parallely by a GPU core which makes aggregation-based analytics incredibly fast. Even when using a commodity GPU hardware, the response time is less than 100ms for tens of millions or records. Furthermore GLayer can directly produce a graphically-rich output (such as heatmaps).
GLayer server is written in Kotlin and uses LWJGL as a GPU interface. Glayer is designed as platform independent, however the target device for testing is NVidia GPUs.
GLayer API
GLayer uses REST API which enables basic data operations such as configuring data stores, setting up data types and configuring projects which includes visualisations. The API is described using Swagger.
Frontend
On top of the RestAPI there is a frontend part which serves these purposes:
- Datastore configuration: data connectors can access data from permanent storages such as SQL databases or csv files. Currently JDBC-based databases and the Clickhouse database are supported.
- Project configuration: project can define how the dataset is going to be visualized and filtered. This includes data type definitions, aggregation strategies, filtering capabilities and cartographical outputs such as choropleth maps, heat maps, histograms etc.
- Project visualization: Once the project is configured, it can be accessed through a permanent link via a standard web browser on a desktop or mobile. Under the hood the UI performs rest queries so that the UI can be possibly replaced by a custom one.
Installation
- GLayer server is distributed as standard windows/linux installation packages
- Google Cloud or AWS AIM image for GPU instance (recommended for p4 instances)