One key objective every commodity business needs to know before starting with AI/ML

Nate: Norbert, you've seen the survey from Commodities People. Twice as many respondents are interested in AI and ML than with data management. What do you think is going on there?

Norbert: Nowadays it's all AI, it's hot. And that's because - only recently - AI became accessible to the consumer. Well, that includes business people too. They are consumers in this context. With ChatGPT in their hands, they are starting to think “but what can this do for my company or daily activities?” – They see the possibilities.

But most are skipping a key step. They don’t fully appreciate the fundaments that any AI or Machine Learning model requires.

Nate: What specifically are most people missing in this respect?

Norbert: Data management. People tend to think in a simplistic way about data. They say ‘o.k. I have a heaping pile of data. I’ll just throw it into the algorithm and the AI will sort it out for me’. And who could blame them? Look how friendly and human sounding ChatGPT is.

That's not how it works.

Nate: So, you're saying as a commodities company - you can't just think: ‘I've got 115 Excel spreadsheets, 3 CTRMS, and a few other things that track my positions across different markets... The AI is smart, it will sort through all the varied data structures and figure it out.’

Norbert: Yeah, that’s not how it works for sure. Maybe that's something Laurent can expand on?

Laurent: There are multiple ways of looking at this topic, depending on what you want AI to ‘figure out’. When people talk about AI, most of the time they are referring to machine learning models. - In the end, you always need a model that is trained on data to solve a certain problem.

Training a model from scratch requires serious investments in terms of time, energy, money and most importantly high-quality data.

Let’s use ChatGPT as an example. It’s a large language model which most people are familiar with by now. The ease of use along with the human touch with which the model responds to queries – the result of training on massive language data sets – make it seem like ChatGPT can do and solve anything. But that is an illusion. Such a model is very good at predicting text, but not necessarily at calculating financial risk for example, at least not yet.

High quality output requires high quality input, therefore, to leverage the power of artificial intelligence in the commodity industry, data management is a must.”

Nate: That’s interesting, but maybe you can clarify something for me. ChatGPT or MidJourney, how exactly do they differ from a potential AI in the commodity industry?

Laurent: The required data for commodities, such as purchases and sales contracts, to train these models is obviously not public. Your positions, your contracts, your market prices. That’s all internal. Some data such as historical financial trading data is public, derivative prices and what not. And that data is clean. But commodity businesses need a lot more than that to build a model.

We see that data landscapes in the commodity industry are often inconsistent and unstructured. This needs to be addressed before feeding the data into machine learning models. As they say: ‘garbage in, garbage out'.

Nate: So let's say you have clean commodity business data, you did that leg work. What are the possibilities?

Laurent: Quite a few.

Most importantly, you must define exactly what problem you want to solve. That is something AI is not (currently) capable of doing for you.

However, looking at we already do for clients. We provide global insights on purchases and sales on a daily basis. … stocks, derivatives, hedging, market data, mark to market, plant performance and so on. All using clean, normalized harmonized data. So not only is this a valuable business function… I call it an ‘AI enabler.’

And the longer a client uses RadarRadar, the more clean, historical data you end up having. This sets you up very well to train a model.

Nate: And those possibilities?

Laurent: Well to start we’re really not that far off from a client being able to ask, “what is my soybean position in Brazil” and get an answer. But we see a lot of potential in ‘outlier detection’ as well. Where the model would flag a trade or contract that doesn’t fit for some reason -> possibly alerting clients to either a mistake or a trend of some kind.

Building on that, we could possibly enable more complex chat queries like “Is my position off?”, “What is the performance of this desk vs another desk? Or how effective were our hedges?”

Nate: Ok, so you two have laid out clear reasons for this mental gap between data management and AI and that AI is essentially inevitable for this industry. What about in the near term? Is data management a long term investment only?

Norbert: What is interesting about AI/ML’s future applications in commodities is that it pays to invest in the infrastructure today. It’s that rare win-win where future proofing also gives you a distinct competitive edge today. Data management alone has a sizable ROI day-to-day… the moment it’s implemented well. So, in a sense investing in this theoretical future has tangible benefits immediately.

Proper data management is a no-brainer for any company, small or large, in any industry. More acutely so for commodities given the complex nature of that industry. The current status quo of a messy data lake or no lake at all is very expensive.

It takes mountains of labor to paint a picture for one – multiple Full Time Employees – And its error prone. Plus, it’s not standardized across orgs. Different controllers, risk managers, traders use different tools, models, etc… There is no clarity…and it’s expensive, slow, and often wrong.

Laurent: I would like to put a long tail perspective on that point Norbert made. Good data management will reduce manual workload right away, but the potential for automating countless time-consuming manual tasks specifically in terms of deep analytics in the future is much greater. The AI revolution that we're on the cusp of is exponential.

Nate: So this isn’t like if a person started to go to the gym, working hard today for a result in the future. There is instant gratification if you get on top of your data management that also happens to set you up for AI in the future.

Norbert: Absolutely. And if you're not getting on top of data management today your future is questionable. To hit at your gym metaphor… It’s true that if you don’t take care of yourself, you may not live as long. That absolutely applies here. The companies that take data management seriously today will outlive the ones that don’t. It’s that simple.

The companies investing in data management right now… They will make more money in the short term by managing risk and margin better. When that AI revolution Laurent mentioned matures… Companies with clean data will be ready. The rest will languish.

Nate: That’s a scary thought. Though not that scary when you consider that it really doesn’t take that long to get on top of your data…That is if you know how. We do.

Norbert: You’re not wrong. For most companies – if we started to implement RadarRadar today, they would have clean data and instant position reports by Q2 2024 – and much more likely to exist in Q2 of 2034.

Nate Chaffetz, Sales Manager