Connectivity to Data Sources
Lyftron can connect to over 200 data source types like relational, MPP, NoSQL, SaaS, CRM’s, or Social Media and supports data access technologies like ODBC, JDBC, and ADO.Net. Click on Connection to know how to connect.
Click Data Sources to see all Source connectors.
Emulation of SQL Server
Lyftron's emulation of Microsoft SQL Server allows any client supporting connectivity to SQL Server to connect to Lyftron.
Emulation delivers compatibility with the following SQL Server features:
-
Metadata model exposed by SQL Server, including metadata catalog, system views, stored procedures, and functions
-
SQL dialect supported by SQL Server
-
Tabular Data Stream network protocol as described in Microsoft TDS documentation
-
Data types and conversions, where all data types are normalized automatically into equivalent SQL Server data types
-
Procedural programming constructs: IF, WHILE, DECLARE
-
Temporary tables (stored in-memory)
-
Selected SQL session environment variables
-
Security model supported by SQL Server
-
Authentication using Windows Integrated Authentication and SQL Server Standard Authentication (Mixed Mode)
-
Job scheduling model and accompanying stored procedures
Click SQL Guide for more details
Built-in Spark
Apache Spark engine is built-into Lyftron. No configuration is needed and a standalone instance of Spark is ready to use for querying and data storage. To utilize the power of Spark, uses do not have to learn to code in Scala or SparkSQL, because everything is wrapped and hidden by Lyftron's emulation of Microsoft SQL Server.
Click Spark Standalone for more details.
Columnar processing
Lyftron is a columnar data virtualization engine. Incoming rows of data are converted into columnar format for high parallelism and high performance processing. Outgoing results are converted into rows for compatibility with the SQL Server protocol.
No ETL
Comparing to classic, flow-based ETL tools, Lyftron implements ETL responsibilities differently. Most of processing requirements can be expressed using SQL and Views and that is the approach in Lyftron.
-
Virtual Databases are wrappers over a data source to provide direct access to data, or created over one many supported data processing engines, that may be used for data integration and caching.
-
Views may depend on each other and blend data using full power of SQL language.
-
Views created in Lyftron’s virtual databases are materializable (aka. cachable).
-
Many processing-engine dependent strategies can be utilized, for example in-memory caching or persistent caching using table rotation.
-
SQL Server’s restrictions of so called “indexed views” do not apply to Lyftron.
Data Security
The untrusted connection is defined in Lyftron as a target which the data cannot be pushed to “as-is”, but encrypted beforehand. Lyftron automatically encrypts the data for you while keeping the data types untouched.
Click Data Security for more details
Query Retargeting
The use of pre-computed aggregate data is a technique to address performance challenges in Data Warehouse systems (DW). An aggregate retargeting mechanism redirects a query to an available aggregate table(s) when possible.
Objectives of Query Retargeting:
- improve the performance of Data Warehouse and Data Marts
- improve architecture flexibility
- allow for a delay of optimization decisions until the performance requirements are fully known
- allow for ad-hoc optimizations for quick reaction to performance issues in production
Click Query Retargeting for more details
Flexible deployment model
Lyftron is fully web based. No installation required. Lyftron comes with the following options:
- Self Option in the Lyftron cloud, which cold either in Azure or AWS region around the world
-
on-premise, using physical or virtual machine with Windows (server and desktop)
-
cloud, IaaS
-
hybrid (mixed), on-premise and cloud
Databases
Lyftron has the capability where you can create a virtual database.
For more details click on Add Database , Manage Database
Schemas
For each database, you can define the set of schemas you want to work with.
For more details click on Add Schema, Manage Schema
Table
Tables can be imported from source databases using Metadata import.
For more details click on Add Table , Manage Table, Bulk Managing .
Views
This section describes the steps required to create a view for the virtual database. You can define a view always in the context of the previously selected virtual database.
For more details click on Add View , Manage View or Bulk Managing
Indexes
Lyftron allows defining an index on a View if the underlying engine supports indexes. Managing indexes in Lyftron is available through GUI and SQL statements like CREATE / DROP INDEX.
For more details click on Add Index, Manage Index
Bulk operations
Bulk operations allow you to perform the same action for many views and tables at the same time. When you select at least one view from the list, the new menu with the Bulk operations appears.
Click Bulk Operations for more details
Management Tasks
Management tasks are object-type specific actions that can be executed against selected object. For example, a "Customers" view, depending on an underlying processing engine, may contain task like "Rebuild persistent cache", but no "Evict from Spark in-memory cache".
Click Management Tasks for more details
Primary Key and Foreign Keys
Primary and Foreign Key capabilities of Virtual Tables and Views within Lyftron.
When importing source meta data any primary keys will be imported. Foreign keys will also be created for the tables if the referenced table is also being imported at the same time.
It is possible to create and edit primary and foreign keys after the meta data has been imported.
Primary and Foreign keys can be used by tools that connect to Lyftron to optimize queries. For this reason, it is also possible to create Primary and Foreign keys for Views also.
Primary and Foreign keys are constraints that are enforced within Lyftron. It ensures that the columns that are primary keys or are referenced by a foreign key column mapping cannot be deleted.
For more details click on Primary Key and Foreign Keys
Tagging
It allows you to tag Databases, schemas, views and tables for easy access to the data. It helps is removing data silos.
For more details click on Tags.
Notification
This section will guide you on how notification works in Lyftron.
For more details click on Notification.
Logging and Debugging
Use logfile and verbosity connection string in your connection settings to handle the logging details only when you want to debugging.
For more details click on Logging and Debugging
Json/Xml Parsing
This section will guide you how Lyftron helps you to parse Json and Xml paths automatically into relational format
Click on Json_Value, lyft_detect_json_value and lyft_detect_xml_paths for more details
Azure Cognitive AI