How two professors and a local consultant teamed up to create a new business that uses machine learning to sort and classify satellite images
It happened, as these things so often do, almost by chance.
In 2013, Winnipeg business consultant Grant Barkman was looking to build on his company’s capacity to help private and public sector organizations achieve their corporate goals.
That’s when he first noticed that some new developments in the field of computer science were starting to have a ripple effect in the world of business. Increasingly, consultants were starting to talk about the value of using machine learning and artificial intelligence to solve common business problems, including how to increase sales.
Barkman, who is President of DecisionWorks, was more than a little intrigued by the idea.
Although he graduated from the University of Manitoba’s Asper School of Business in 2009 with a Masters in Business Administration (MBA), Barkman was also fairly well versed in the concept of machine learning and neural networks, having earlier received a degree in computer science. As a result, he understood that these new technologies could have a transformational impact on business.
And so, he decided to seek a potential partner in the field.
“I started looking for who was doing best practice,” says Barkman.
In short order, he was introduced to Chris Henry, a professor at the University of Winnipeg who specializes in applied artificial intelligence. That meeting would mark the beginning of a mutually beneficial relationship, one that would lead to the creation of Deep Geo, a company that uses machine learning to analyze satellite images.
However, the story of Deep Geo is more than the tale of a high-tech start-up. It is also a good example of what can happen when entrepreneurs work with researchers to develop new answers to old problems. And it illustrates how timely investments in university-based research can play an important role in enhancing Manitoba’s economic development.
As one might expect, Deep Geo was not created overnight.
Initially, Barkman and Henry decided to team up to tackle a number of other projects that involved using machine learning to advance business goals.
One project, which was supported by a federal grant, involved helping a credit union better market its financial services. This initiative focused on using machine learning to analyze data that would help the credit union understand which customers would be more likely to buy a particular financial product, such as a mortgage, a loan or a guaranteed investment certificate (GIC). This, in turn, would allow the credit union to target its marketing efforts.
“Rather than try to shot-gun market to 30,000 members, they could personally phone 150 and get a higher buying ratio,” Barkman explains.
As their relationship evolved, Henry pitched Barkman on a new idea.
Henry had been working with Chris Storie, a professor of geography at the U of W, on a method for applying machine learning to the analysis of satellite imagery.
“They were taking satellite images and doing machine learning for recognition of features like land elevation changes, tree structures, health of trees, different types of wetlands, water quality analysis – all of these things could be done through satellite imagery,” explains Barkman.
The idea had tremendous potential. Government departments like Forestry Manitoba and Geoscience Manitoba (a section within Agriculture Manitoba) have mandates to create updated relief maps of the province. Corporations like Manitoba Hydro have a need to carry out wetland and water quality analysis. The application being developed by Henry and Storie would make these tasks easier and less expensive to carry out because they could be done by analyzing satellite images as opposed to hiring a plane to fly over and photograph the area that needed to be surveyed.
“They (the provincial departments) had a legislated requirement to produce these maps every year, but they were producing them once every five to seven years because they couldn’t afford to actually produce these updated maps,” says Barkman. “We could could do it with satellite imagery more accurately at a 1,000 to one reduction in costs. The return on investment (ROI) is just phenomenal.”
And so, in the spring of 2019, Deep Geo was born.
“Chris Storie is the data prep guy, Chris Henry is the data use guy and we’re the commercialization partner – that’s the arrangement,” says Barkman.
The partnership represents one of the first commercialization deals of its kind for the university.
“The U of M has been doing that (commercializing ideas developed by its professors) for a long time,” says Barkman, noting that the U of W has been less active in this area.
The concept of machine learning has been around for a while. It essentially involves creating a computer algorithm, which is based on a mathematical model, to carry out certain functions. If you feed the algorithm enough data, it can begin to sort and classify the information according to various parameters.
Until recently, the capacity of these kinds of algorithms was limited by computing power and data sets. But all that changed soon after Henry completed his PhD in 2010.
As he explains, advances in computing power, along with access to huge archives of data such as satellite images, made it possible to develop algorithms that could “learn” how to sort and clarify massive amounts of information quickly and efficiently.
“The introduction of big data, the introduction of high performance computing, and then the introduction of machine learning algorithms that were bigger and had more parameters than was previously computationally possible – those three things came together in 2012 and it was a game changer,” says Henry. “All of a sudden, you could start solving real-world problems with machine learning.”
As Henry and Storie compared notes at the university, they started talking about how they could work together to use machine learning to sort out satellite images of the province’s geography.
“We realized that in one aspect of our research, we were both doing the same thing, which is image segmentation… Chris was doing it from computer science, and I was doing it from geography,” says Storie.
Simply put, image segmentation means assigning a value to every unique aspect of an image.
“It’s almost like doing a reverse paint by numbers,” says Storie. “But in the case of satellite imagery, all your green space becomes a category, and all your urban space becomes a category,” he says.
In many ways, geography and machine learning are a natural fit. Geographers looking to keep tabs on land use changes can tap into a huge database of Landsat satellite images of the earth that have been compiled by the United States Geological Survey since 1972. And massive amounts of data, of course, are what is needed to “train” an algorithm to sort and classify images.
“That is the story of machine learning,” says Henry. If you have enough data, and you have enough computing power, you can build an algorithm that will eventually become very good at recognizing and classifying the patterns in the examples.
Henry, Storie and a team of about 10 U of W students (including computer coders and image analysts) spent about six months developing their first algorithm for sorting and classifying satellite images. They then spent another year or so refining and improving the algorithm’s capabilities.
Eventually, they ended up with a powerful algorithm capable of sorting through thousands of satellite images to produce land use/land cover maps that can reveal various changes on the ground, including land use, water levels and even the health of forests. These maps can then be used to help the government manage the province’s natural resources.
“We can identify and monitor where logging companies have cut down large swaths of (trees). We can monitor fire, like what is happening in Australia. So we can map the extent of the fire and, by repeating the mapping afterwards, watch how (the forest) regrows… We can give it to water resource managers and say, ‘Here is where all your water is right now.’”
However, Storie says the algorithm can only produce what it sees. Each pixel on a Landsat image represents 30 metres by 30 metres on the ground. “So we can see the forest, but unless the tree is 30 metres by 30 metres” it is only seen as part of a large group of trees.
Although still in its infancy, Deep Geo is already working with a number of clients through its partnership with DecisionWorks, including Manitoba Agricultural Services Corp., Forestry Manitoba, and the City of Winnipeg.
Deep Geo is also involved in a project to build channels connecting Lake Manitoba and Lake Winnipeg, a move the is designed to help ease flooding.
As Barkman explains, DecisionWorks is working with First People’s Development Inc. to train between 10 and 15 members of five First Nations communities in the Interlake area to use drones to gather land and water images around the areas affected by the project. That data will then be analyzed by Deep Geo.
“Deep Geo uses satellite imaging where we can, but where we can’t we do infill with drones,” says Barkman.
In addition to providing some residents with training on using drones to gather geographic data, the project will provide First Nations communities with more oversight on the channel outlet project.
“This will allow them to address issues that may arise, including soil erosion or degradation of water quality,” he says. “These are all things that have real meaning to First Nations.”
While Deep Geo is well on its way to becoming a success story, Barkman says it is important to remember that machine learning technology can be used to solve all sorts of business problems.
For example, he says he recently helped a Manitoba powdered milk company expand to China using machine learning to help develop a plan for expansion.
The company started in Manitoba with one mixing machine used to create the product and about 3,000 square feet of space. In time, Barkman was able to use machine learning technology to help calculate how much capacity they would need to serve the entire Canadian market.
“So we move them to a new facility, 20,000 square feet and three mixing rooms. Then they came to us and said, ‘Okay, we want to expand to China. Wow! Okay, now you want to go to a market that is probably $5 million revenue to a market that could be potentially $1 billion.”
Needless to say, trying to determine how to tackle a market that size from scratch is a daunting challenge, one that would be beyond most people.
“There is no way that you can possibly forecast and predict the complexity of that market dynamic that is going to allow you to create capacity and capability within a facility here to scale appropriately to that market,” says Barkman. “You have to match your marketing strategy with your capacity strategy, and that’s way beyond a human being’s ability to understand.”
Thanks in part to the insights provided through machine learning technology, Barkman says the company in question has made inroads in China, and is expecting to enhance its market share in the years to come.
Of course, it is important to remember that the work being done today by DecisionWorks and Deep Geo is only possible because of two things: investments in the research being done by Henry and Storie, and the subsequent decision to commercialize the idea.
In the early days of their research, Henry and Storie received funding from several provincial government departments through Geoscience Manitoba, which could see the potential benefits of their work. But Henry and Storie knew they would not be able to commercialize the idea on their own. As they explain, they are professors, and their job is to carry out research and train students, which leaves little time for anything else.
“The problem is that commercializing technology and running a business requires a time commitment,” says Henry. “That’s where DecisionWorks comes in because they understand machine learning and they understand everything that is involved with actually bringing (a good idea) to market,” he says. “Without the partnership, I don’t think the commercialization would be possible because we just don’t have the time to do it. So it is a nice symbiotic relationship.”
That collaborative effort is paying off in in many ways, says Storie.
In addition to creating a made-in-Manitoba business capable of providing a service that was not readily available at that time, Deep Geo is also helping to train students in a new technology.
“As we (become successful), the people at Deep Geo doing the work… are going to be the students that we trained in many cases,” says Storie.
In that sense, the story of Deep Geo also underscores the important role universities play in economic development. While their primary job is to educate students, they also possess the expertise and research capability that many businesses require to remain competitive in today’s economy. As a result, private sector companies will often approach universities with problems that require more research capacity than a company may possess. That makes sense, says Storie, because it is often cheaper to invest in university-based research to solve certain kinds of problems than it is to build internal capacity that might not be required over the longer term.
“We’re much better equipped on campus in terms of research technologies and human resources in terms of hiring students to build research teams to solve problems,” says Storie.
Indeed, Storie says he has attended a number of meetings over years involving companies looking to set up shop in Winnipeg. Often, they would be attracted to Winnipeg for a variety of reasons, including its central location. But during these meetings, it wouldn’t take long before the conversation would turn to the local universities and what they could provide in terms of access to academic expertise and trained students.
“Then the question becomes do we have the software engineers, do we have the linguistics or whatever their needs are?
Companies also like investing in research projects at universities because they know the students who work on them will one day be available to join their workforce. In other words, a dollar spent on university-based research is also dollar spent on the training of a potential employee.
“The multiplier effects are immense at universities, and corporations like that,” says Storie.
The bottom line is this: If Manitoba businesses are to continue being successful, they will need to have access expertise and a highly skilled workforce.
Universities are well positioned to provide both of these things. As the Deep Geo experience shows, timely investments in research can lead to the creation of new ideas. Collaboration between researchers and industry can result in partnerships that will help ensure these new ideas are turned into commercially viable products or services.
Needless to say, all of this activity benefits the economy, not just by creating new knowledge-based jobs, but by helping to train the workforce needed to fill them.
