This post was written in collaboration with Jason Labonte, Chief Executive Officer, Veritas Data Research
In the realm of healthcare and life sciences, data stands as the linchpin for propelling medical breakthroughs and improving patient outcomes. Utilizing the right real-world data source can be a catalyst for innovation across healthcare, research, and pharmaceutical organizations. According to Gartner, leaders in data and analytics who engage in external data sharing can generate three times more measurable economic benefits compared to those who do not.
The Vital Role of Mortality Data
Mortality data is a critical cornerstone in health analytics, offering profound insights into treatment efficacy, public health policy, and protocol design. Yet, capturing these crucial endpoints is a challenge within conventional clinical datasets like insurance claims or electronic health records. This gap necessitates augmenting clinical real-world data (RWD) with a mortality dataset to accurately understand patient outcomes.
Veritas: Pioneering Quality Mortality Data Solutions
Veritas is resolving the scarcity of reliable mortality data. Founded by industry experts, Veritas employs cutting-edge technology and streamlined workflows to aggregate, curate, and disseminate foundational reference datasets. The process involves meticulous data ingestion from diverse sources, refinement using third-party reference data, and the creation of a comprehensive Fact of Death index.
Datavant Streamlines Insight Generation via Databricks
Enter Datavant, a key player in reducing data sharing hurdles in healthcare through privacy-centric technology that enables the linkage of patient health records across datasets. Their collaboration with Databricks stands as a testament to advancing seamless data sharing in the healthcare industry. Veritas leverages the Datavant technology to tokenize and de-identify their data to be shared with research, life sciences, insurance, and analytics organizations looking to better understand patient outcomes.
Datavant’s Innovation on the Databricks Platform
Datavant introduced its Tokenization Engine tailored explicitly for the Databricks Platform, eliminating the need for custom deployments or maintenance. This library, designed for Databricks workspace, harnesses the power of Spark technology for enhanced performance. Notably, it supports direct reading and writing to locations in lakehouse, streamlining data pipelines for efficient token generation.
Accelerated Efficiency: Veritas’ Journey with Datavant on Databricks
The integration with Datavant on Databricks proved transformative for Veritas, simplifying implementation, reducing processing times, and reducing costs.
Implementing the Datavant on Databricks was a simple installation of a python wheel. This process required less effort to set up data pipelines and was running within 1 day!
Previously, Veritas executed tokenization, transformation, and uploading in about 4 hours for 80 million patient records. Leveraging Datavant on Databricks and the power of Databricks’ Spark technology, Veritas witnessed an astounding 80% time savings. They accomplished the tokenization of 80 million records in just 20 minutes, followed by transformations in 5 minutes, and delivery in 15 minutes.
The Datavant on Databricks product limits data movement with tokenization happening within Vertias’ Databricks Workspace. Veritas saw not only time savings, but cost savings as well.
Veritas leveraging the partnership between Datavant and Databricks signifies a shift in the speed-to-insight, which will ultimately drive innovation and transformative advancements in the realm of life sciences and healthcare.
To delve deeper into these pioneering solutions and their impact on revolutionizing life sciences data sharing, check out the following resources:
