Quod Erat Demonstrandum
Posted: December 20, 2016
By Alan Yates, IEA Principal Consultant
How the IEA uses Data Analytics to improve your business
Here at the IEA we like to talk about demonstrator projects. These are a core part of our exciting workload and provide clients with an opportunity to de-risk their investment into new products and services, or potentially to explore how data analytics can improve their business operations. In our world-view a demonstrator project is usually a 3-6 month proof-of-concept where we try to apply our expertise to a business problem to show the “art of the possible” using novel data analytics techniques. It’s normally a preliminary step prior to seeking funding or further investment – a way to help a client build a case for further action.
We like to think of a demonstrator project as having 3 stages – Discover – Explore – Demonstrate.
Discover – understanding the data challenge
In the discovery stage we clarify the problem that our client has presented to us. By asking questions such as, “Why is this important?”; ”What decisions are you looking to improve?” and “If we were successful what benefits would a solution bring?”, we help to understand the underlying drivers and objectives of the client. We also do a rapid literature review to survey the important issues and drivers in the client’s sector and try to find out what the state of the art is in academic research and commercial product offerings. Often this stage can be a significant proportion of the overall project duration as we seek to ensure that we have the required data to do our job and also that the client is clear about what they want to get out of it.
Here are some recent examples of things my colleagues and I have had to become ‘expert in’ at very short notice over the last year – shrimp farming; phenological crop models; renewable energy; metaldehyde impacts in drinking water; flood modelling; handling NetCDF files; the physiology and behaviour of tuna; numerical weather modelling; data assimilation; economies of small island developing states; radar and optical satellite sensing; biodiversity databases; the oceanography of the Indian Ocean; ecosystem modelling; commercial fishing techniques; global ocean earth observation datasets; basic programming in python; supply chain management; and ecosystem services.
So variety is the spice of life here, which I think is great. I expect the list to be twice as long next year!
Explore – datasets, variables and modelling
The exploration stage typically involves cleaning the required datasets and extracting the features and variables that are needed for modelling. A feature is something that can be derived from the raw data and which can then be used as a relevant input to a model. For example we may wish to derive from a numerical weather model the presence of a tropical storm, or from weather observations the existence of a drought. This cleaning and feature extraction task is normally an effort-intensive process that occupies the majority of the time.
Typically we then then merge the input data sources in order to analyse them and often produce a predictive model. The predictive model is normally used to inform a decision that our client has to make. Often in our work this is do with identifying and quantifying risks: What is the risk of flood in the next 48 hours?; What should we do to maximise our chances of getting a good fish catch?; What is the likelihood of fog occurring in the next 2 hours?; What environmental risks should a construction project be mitigating?
Demonstrate – data visualization
Finally, we need to take the outputs from our models and show how they can be put to use. Often there is a lot of data visualization to be done and the building of a web-based application to show how the models could be used in practice. We like to demonstrate how our models and solution ideas would work against real historic data, to show some potential value. Our last task is to complete a final report and present back to our client with a demonstration of our findings.
Often at early stages we are searching for example ‘use cases’ where there are clearly demonstrable benefits. More often than not these tend to become apparent, but there remains a long journey to further develop the concept to commercial viability. Our demo project can thus be seen as a valuable first step in a client’s journey to gain some value from environmental data analytics. And as Mark Twain once said: “…the secret of getting ahead is getting started…”
Got a challenge and need to know if data holds the answer? Contact us.
Read about our projects here.
Catch up with our latest news.