Spark

The Lyftrondata Provider for Spark offers the most natural way to access Spark data from Lyftrondata with ease and also enables to connect with BI, MDM & ML tools, Data warehouses, Databases and other SAAS based applications with zero code and zero infrastructure requirements. The provider wraps the complexity of accessing Spark data into easy-to-integrate relational fully managed ANSI Sql format. Make faster and better business decisions with Lyftrondata’s Spark data provider and automatically build your data migration pipelines in minutes, not months

The provider hides the complexity of accessing data and provides additional powerful security features, smart caching, batching, socket management, and more.

Key Features

• Comprehensive Delta load mechanism.
• Real-time access to Spark.
• Comprehensive full support of ANSI Sql to query data with ease.
• Collaborative query processing.

Prerequisites

The user must have credentials for Spark, Lyftrondata and your destination data warehouse, lake or database to perform the data pipeline operation with Lyftrondata

Establishing a Connection with Lyftrondata's Quickstart Steps

Create your Spark connection with Lyftrondata by following the 5 easy steps show below:

Step1. Add your connection

Click on Connect section on the left panel →  Click on Add Connection button

Step2. Select your connector

In the connector selection panel, search and click Spark for your connection 

Step3. Enter your connection details

In the Connection String section enter the values of the below parameters. The following connection string is required to establish Spark connection with Lyftrondata.

Server=[your_server];Port=[your_port];

Key	Value	Field
Connection Name	Enter your connection details	Required
Server	127.0.0.1	Required
Logfile**	Use the logfile option to debug your job and provide your connection name to generate the log file.  [ConnectionConfigurationPath]\Connection_name_log.tx	Optional
Verbosity**	Choose verbosity 1-5 based on the severity of debugging	Optional

** For more information, check the Lyftrondata logging and debugging section.

If you want more detailed information about how to establish a connection with Lyftrondata, click on Lyftrondata Connection Quick Start guide.

Step4. Test your connection

Once you are done entering your connection details, simply click on the Test Connection button to test the connectivity. In case your connection fails, add Logfile and Verbosity parameters and check the Lyftrondata logging and debugging section, to debug the error.

Step5. Save your connection

Hurray! Now you have successfully connected with the Lyftrondata Spark connector and can utilize the connector to Extract, Warehouse, Analyze, Visualize and Share your data.

 Advanced Settings