DataPelago Documentation

Congratulations! Thanks for purchasing the #1 accelerator for Apache Spark environments. DataPelago is redefining the world of data by building a differentiated solution for transforming the economics of processing data of any scale or structure at performance and cost unsurpassed in the industry.

This documentation will walk you through the basics of how to get started with DPA-S. If you need any help, we are always here. Just get in touch with us at support@datapelago.com.

Contents

• Home
• What is DataPelago Accelerator for Spark?
  • Use cases
• Architecture
• Sign up for DataPelago
• Deploy on GCP
  • Supported Versions
  • Prerequisites
  • Deploy on Dataproc Cluster
    • Create cluster using Command line
    • Create Cluster using Dataproc UI
  • Deploy on GKE
  • Configuration
    • Spark Settings
  • Using the Cluster
  • DPA-S Memory Configuration
  • Limitations
    • ANSI Mode
    • Case-sensitive Mode
    • Regexp Functions
    • File Source Format
    • JSON Functions
    • NaN Support
  • Observability
• Troubleshooting
  • 1. Finding DPA-S Plugin Version and Environment Details
  • 2. DPA-S Load Error
  • 3. Corrupted or Incompatiable DPA-S Release Package Files
  • 4. Insufficient Privileges to Access or Run DPA-S
  • 5. License Verification Fails
  • 6. Invalid Subscription ID
  • 7. Segmentation Fault Errors in DPA-S Components
  • 8. Find the operators accelerated using DPA-S
  • 9. Queries Running Slowly
• Deploy on AWS
  • Supported Versions
  • Prerequisites
  • Deploy on EKS Cluster
  • Instructions to run your Spark Application with DPA-S Container
  • Upgrading or Patching DPA-S
  • Observability
  • Troubleshooting
• Video Tutorials
  • Dataproc Deployment
  • GKE Deployment
  • Performance Demo
• Open Source Components