Do you have a question? Want to learn more about our products and solutions, the latest career opportunities, or our events? We're here to help. Get in touch with us.
We've received your message. One of our experts will be in touch with you soon.
Cawthron Institute scientists help to grow and protect New Zealand's aquaculture industry, improve water quality, and investigate algae and bioactive substances that could be used in everything from pharmaceuticals to food production.
Celebrating its centenary in 2021, Cawthron remains New Zealand's largest independent research institute. It is also investing in its future, a future increasingly driven by data-intensive science.
“Cawthron’s real strength is extracting scientific value from data in many different ways, shapes, and forms,” says Darren Wood, Cawthron Institute’s management information systems (MIS) developer.
“A few years ago, we realised that our existing infrastructure was limiting our ability to share and publish data online for our clients and even for our own researchers.”
At the Cawthron Aquaculture Park, dozens of Internet of Things (IoT) sensors gather data in real time from the institute’s research hatchery. In nearby research offices, scientists perform genomic analysis on DNA sequencing data from fish and shellfish species, producing large batches of digital files in the process.
“The amount of data we are producing is just growing exponentially,” says Nick King, Cawthron’s senior scientist, shellfish aquaculture, and the man who helped design the research hatchery.
"We didn’t have a great system for easily making sense of that data, allowing remote access, and effective data back-ups.”
What Cawthron needed was a data platform that would make it easy to share data among researchers and clients swiftly and securely, while leveraging the institute's existing IT infrastructure and automation networks.
That's when Cawthron discovered data lakes.
“We had an initial conversation with Datacom. I asked them if they could show us what a data lake was,” Nick remembers. “That’s where it all started.”
So, what is a data lake?
It's a way of storing data drawn live from multiple sources in its original format and then making it available to scientists in the way they need it. It's fast, flexible, and means scientists have a direct feed from sensors and other producers of data to update their research in real time.
Datacom has migrated Cawthron’s data assets to the cloud with the use of the Microsoft Azure platform. Data from Cawthron’s IoT network is now received by the Azure IoT Hub and fed straight into Azure Data Lake.
From there, it can be configured in any format required by researchers or clients. Power BI and Azure Stream Analytics allow staff to get real time updates on the status of the institute’s data assets through a visual dashboard.
“It’s really allowed us to lift our game,” Nick says. “We can see exactly what the conditions are for an experiment without having to run out to the hatchery with a thermometer.
“If we want to put smart buoys out in the Marlborough Sounds to collect data from mussel farms and the ocean, we can suck that data into this data platform without any extra work.”
The ease with which data can now be gathered, accessed, and analysed is saving Cawthron precious dollars that it can channel into more research.
“We will realise cost savings, not just operationally, but in freeing up our people to focus on where the value really is,” says Darren.
A collaboration with Datacom spanning several years has resulted in a scalable data platform that is not only making life easier for Cawthron staff and researchers but facilitating more international research collaborations.
“It’s common for the work to happen across multiple organisations”, Darren explains.
“With some projects, you’ve got these terabyte-sized data sets that you're shifting around and working with. Being able to do that securely using this platform is transformational.”
With Azure in place, the Cawthron team is exploring the future potential uses of the platform, including supporting large scientific models and collecting libraries of acoustic files, such as dolphins communicating with each other, as well as analytics for video feeds taken from cameras on buoys.
“We couldn’t have achieved a fraction of this potential without the help of Datacom,” says Darren."We had the broad reach of expertise within Datacom to draw on but the local touch too with Matthew Polson and his team here in Nelson.
“What we’ve built here is integral to the future of both our scientific and commercial endeavours.”
For Nick, the work is helping to realise Cawthron Institute's purpose.
"Our business is knowledge and data is the raw material. So, we need to create these factories within our organisation to turn those raw materials into knowledge and that's what this platform helps us do."
