Very promising papers come out every day about machine and deep learning algorithms used for medical and healthcare purposes and this field is just blowing up. I received questions from physician colleagues who are not A.I. experts about how they should evaluate these papers. I thought I would share a few things I do myself. - It matters where the paper is published. It's not scientific elitism but it matters if a new algorithm gets presented in Nature Medicine or in a third level journal. - Look for the Methods section where they describe where they got the data. No algorithm can be trained without a good amount of quality data. Some groups just use a good dataset and do tricks on it to make it bigger (e.g. invert images to double the database size). That's not the same as getting a huge amount of annotated data. - Look for the clinical collaboration they had. If an algorithm works well on a pre-selected dataset, that sounds fine. But when it works on clinical datasets too, that is the one we are looking for. When I saw @ DeepMind collaborating closely with the NHS, I knew that was the right direction. Can you please share the tricks you use while assessing a medical A.I. paper? #digitalhealth #healthcare #medicine #technology #AI #artificialintelligence #TMFchessjourney …see more