How to make the most of Data Science

One of our clients has a team of over 15 full-time Data Scientists. This team was full of talented, intelligent scientists, but they were slow to produce models and results. For instance, it took more than eight weeks to build a model that ranked customers who were most likely to buy a new product or participate in a loyalty program. Every time there was a new product added to the catalog – may be because of a seasonal event – they restarted their Data Science life cycle to retrain and push a new model to production systems.

The client believed that these models should only take a few days to create, and they were right.

We delved deeper into this and realized there were two main problems. The first problem was that new datasets were required for each new model. However, the datasets were messy and required significant ETL processing before they were ready to be consumed by the team. We knew that if the data scientists had clean datasets, they could get a model up and running in a matter of hours.

We were asking data scientists to do a job they weren’t good at. It isn’t the job of the data scientist to wrangle, clean, and prepare data.

The second problem was that there was a lack of clarity about what was ‘good enough. Data scientists are all about the quality of the model, tuning hyperparameters and getting two percent more, or improving it by 50 basis points — that’s what gets them excited. However, it isn’t always necessary to build the best, most perfect model. For example, if you have a $100 million renewal opportunity, you may not need to wait until your renewal prediction model works at 99.88% accuracy. It might be enough to move the product into production when you hit 90% accuracy. Getting a ‘quite-good’ product into use quickly might deliver more revenue than waiting to deliver something that is near-perfect.

A path forward

Data scientists are expensive. If you need an agile data science team, it is important you are getting the most value from them. One way you can help is to ensure you have clear definitions of what ‘good’ looks like. The second way is to ensure that the initial work of cleaning, wrangling, and processing the data is completed by a data platform. Here is where ProArch’s Accelerated Data Platform comes into play.

To learn more about our Accelerated Data Platform and how it can improve your readiness to apply data science to make or save millions, click here. If you wish to talk with one of the team, get in touch using the form below.