Generate optimal embeddings for any ML task
Train accurate ML models on limited, sparse, and high-dimensional data without extensive feature engineering by creating statistically optimized representations of your data.
Upcoming Event
See you at NeurIPS!
We’re excited to see other cutting edge ML research this December.
Come say hi!
Don’t let imperfect data get in the way of great models
By learning how to extract and represent complex relationships in your existing data, Dark Matter improves model performance and speeds up training without extensive feature engineering or resource-intensive deep learning — enabling data scientists to spend less time on data and more time solving hard problems.
Case Studies
Customer Conversion
Dark Matter significantly improved model precision and f1 scores in predicting customer conversion in the online retail space.
Cancer Kinase Inhibitors
Model performance metrics improved across the board when trained on an optimized embedding learned from a sparse, high dimensional data set.Customer Churn
Training XGBoost on a better representation of the data improved predictions of customer churn in the banking industry.
Enhance your pipeline, no matter your model or domain.
Slots in Seamlessly
Surprisingly lightweight, Dark Matter represents a transformative new step in the data science pipeline that doesn’t alter your existing processes.
Domain and Model Agnostic
We make any model in any domain better simply by creating richer representations of the relationships in your existing data.
Secure Integration
Integration is available on-premises or via cloud API. Retain total control of your pipeline, keeping the privacy and integrity of your data intact.
Optimize training and inference across tasks
Forecasting
Price predictions
Supply and demand
Customer churn
Recommendations
Ad placement
Content suggestions
Product personalization
Specialized Tasks
Chemical discovery
Sensor data
Virus-host interactions
Seamlessly slots into your existing ML pipeline.
Dark Matter is a surprisingly lightweight solution that integrates seamlessly into your pipeline — either on-prem or via cloud API. So you maintain end-to-end control of your ML process, data, and models.
Securely installs in under 5 minutes
// Import
import ensemblecore as ec
// Authentication
user = ec.User()
user.login(username='USERNAME', password='PASSWORD', token='TOKEN')
Backed by:
Mark Nelson
Former CEO of Tableau
Research
Feature Enhancement: A New Approach for Representation Learning (Whitepaper)
Discover a novel approach to representing complex, non-linear relationships inherent in real-world data.
Feature Programming for Multivariate Time Series Prediction (ICML)
Learn about a new framework for automated feature engineering from noisy time series data.Resources
Blog
Op-eds and thoughts on the state of machine learning and AIDocumentation
Developer support, API docs, quick-start guidePublished Research
Ensemble research, papers, and conferencesFrequently Asked Questions
Dark Matter is available for on-premises installation on your machine and using your compute resources, enabling you to retain complete control over your proprietary data. On-prem deployment ensures that we never see your data.
We encourage you to try Dark Matter with your data and model to compare the results with your existing pipeline. Most customers use it in a testing environment with sample data to minimize resource requirements before putting it into production. If you’d like to set up a trial, please fill out the form here and we will be in touch.
While Dark Matter does create new variables, its mechanics are fundamentally different. Traditional synthetic data recreates existing distributions from Gaussian noise, so no new information is created. This has the virtue of anonymizing data (which is essential for some regulated industries), but it has minimal impact on predictive accuracy as it mirrors the statistical properties of your data.
In contrast, Dark Matter learns how to create embeddings that have different statistical properties and distributions. Using our new machine learning algorithm, it’s able to converge on nearly orthogonal features that measurably improve predictive accuracy.
One of the primary benefits of Dark Matter is that it lowers the barrier to useful predictive performance by creating richer representations of your data. That said, there is a theoretical minimum threshold of data quality and volume that can be useful (i.e. if what you’re working with is mostly noise, it probably won’t help). Our rule of thumb is that if you have a working data science pipeline that’s generating mediocre predictions, Dark Matter can improve its performance.
