Every day, new television advertisements, journal articles, books, podcasts and companies pop up, touting machine learning (ML) and artificial intelligence (AI) as the next wave of change in our modern society. Experts believe ML and AI can answer challenging questions, create novel insights and accomplish arduous tasks that were never before possible.

From Klaus Schwab discussing AI as a key component to future technological advances in his recent book, The Fourth Industrial Revolution, to the largest companies in the world like Facebook and Microsoft investing billions of dollars into data procurement and cloud computing capabilities, there has never been more hype in the data science space.

The algorithms at the heart of ML and AI are not new. Their roots date back to the 1950s, when the concepts behind technologies such as neural networks and natural language processing were born.

Fundamentally, ML and AI algorithms are coding sequences that allow machines to make predictions from data, measure the accuracy (or error) of those predictions, then use that error to refine future predictions.

The machine repeats this process over and over again in an iterative fashion, slowly improving the quality of its predictions or "learning."

Once running, these algorithms can continue on their own indefinitely without human influence, making ongoing predictions while automatically incorporating large quantities of new data to refine their accuracy.

With great advances in computing power, data availability and data storage, as well as easy to understand programming languages created to deploy ML and AI systems, the keys for ML and AI to thrive are firmly in place.

Examples of its successes are everywhere, from facial recognition on our iPhones, to the ability of Amazon to predict what purchases we are interested in based on our prior purchases, to self-driving cars utilizing computer vision to "see" the road in front of them and navigate key obstacles.

Neural networks are particularly good at image interpretation and pattern recognition, and recent medical research has leveraged these attributes to create interesting clinical insights.

For example, a group of researchers at the Mayo Clinic used over one million ECGs of patients with chronic kidney disease who had potassium levels drawn within hours of their ECG recording to train a convolutional neural network to subsequently predict potassium levels from ECG tracings alone with remarkable accuracy.

They next applied their technology to an Apple watch program capable of creating an individual's ECG, to effectively predict one's potassium level from a smart watch.

Additionally, a group from Google AI used a neural network to analyze photographs of retinal images to detect the presence or absence of diabetic retinopathy, with an accuracy exceeding a sample of board-certified retina specialists.

Further, a group of investigators from UCSF and Northwestern used echocardiography clips to predict various cardiac pathologies including amyloid cardiomyopathy and hypertrophic cardiomyopathy.

Perhaps the most astonishing aspect of these models is that once they have been established, they can be repurposed to answer novel questions that may not have even been considered before.

For example, the neural network described above by Google AI to predict diabetic retinopathy from retinal images was also able to predict gender, smoking status, the presence of hypertension and cardiovascular risk, all from the same retinal image. This concept allows innovators to ask questions that may have previously been considered unanswerable.

This is highlighted by a group also at the Mayo Clinic who recently predicted the development of new atrial fibrillation from individual's ECG in sinus rhythm. With the rapid growth of data sources such as advanced imaging modalities, we should expect these advances to continue at an exponential pace.

However, we must proceed with caution and monitor key steps in the process before we can make meaningful progress.

For one, the majority of the published medical literature on ML and AI algorithms takes a large dataset, segments it into a training set and a test set, trains the algorithm on the training data, and then tests that algorithm on the test set.

While the datapoints in the training and test set are unique, this approach fundamentally takes one source of data to both make predictions and test those predictions.

This should be considered hypothesis generating only, much in the way that associations from observational data are. These algorithms need to be further examined with the same rigor we test medical therapies – in randomized controlled trials in diverse populations – before utilizing them to make clinical decisions.

Failure to do so may result in products being available to physicians before they are truly ready for clinical use, much as was the case for IBM Watson for Oncology, resulting in errant recommendations for managing sick patients.

Second, we must be extremely aware of bias in our data. If our data is biased, then so will our predictions be, perpetuating the negative stereotypes and repressive cycles that unfortunately exist in our society.

This was all too clear when an algorithm created by the company Optum meant to help hospitals allocate resources to the patients in the most need and resulted in significantly more resources being allocated to white patients than black patients, even though they had similar degrees of need.

Lastly, we must be hyper aware of our need to protect patient confidentiality and make it clear how, when and why we are using data, where we are getting data from, and who we are sharing data with, as the patients we obtain this data from are its true owners.

While challenges inevitably lie ahead, the ability to use large quantities of data in novel ways while maximizing the potential of advanced computer systems and prediction algorithms has many individuals optimistic for a good reason.

As the field of ML and AI in medicine expands, I suspect we will continue to see how the combination of ML and AI plus humans will create insights we could not have reached by the human mind alone.