Scalability: Could big data help?

People around the world are buying more stuff beyond their own borders. By 2021, Shopify expects that global retail e-commerce sales will reach $4.5 trillion. Those are insane numbers, but there’s no doubt that it will put high pressure on the logistics industry. Twill is looking to adapt with the help of big data.

The main challenge for the coming years will be scalability – not only of our systems and processes, but also of people, ideas and other resources. We could choose to be afraid of what is to come, but those figures will only continue to increase after 2021. So we have to prepare ourselves by putting the latest technologies in place, such as cloud computing services, blockchain and (big) data analytics, and be ready as soon as possible.

And we’re already doing that by adapting Twill to be able to process more bookings every single day, but to us this doesn’t seem to be enough. If we continue to enter other business areas this could result in greater pressure on the platform. So how do we best deal with scalability?

‘We know the importance of big data, but how can we best use it?’

Everyone knows the term big data by now. But only a few companies know how to handle data and use it in the right way.

But for me, we have to remain focused on both big and small data, if we don’t refer to it based on volume. To define small data: all data which is related to a specific person and can be used to adapt your system based on the knowledge you have of the user.

We also have the differentiation between internal and external data. Big data is the combination of both internal data owned by the company and data from external sources. Given the wealth of data available in this scenario, managing data becomes much more complex, but at the same time highly valuable to an organization’s processes.

But there comes the challenge of scalability.

How can you combine all that data and process them quickly in order to best serve your needs? For me, we should think the other way around: what problem do you want to solve and what data do you need in order to do so?

In this way, you leave out any data which is irrelevant to your business. After this you can scale using cloud computing services and state-of-the-art algorithms.

The next step is not always applied, and most of the time it’s forgotten in the process: learn. The output of your system can be used to let the system learn and improve for further outputs. This is what we call machine learning, and can also be used on other types of data analytics.

Process automation across the logistics industry

Within Twill we are constantly looking for opportunities to automate our processes. The main reason for this is scalability.

If we have millions of users, we cannot inform them manually, check the cargo daily and handle delays, etc. We need – no we must – automate the complete logistics chain in order to scale our platform.

Some examples of projects where this is already being done include adding payment services to immediately release your cargo, using external data sources to automate and optimize the process, as well as partnering with other companies to add more value to our customers.

Lastly, the industry needs to be able to provide real-time prices and transit times from all vendors. Combine that with algorithms and small data we hold on each customer (i.e. whether they are focused on price or speed), we can recommend a specific vendor to fulfil their requirements at that moment in time based on their specific, individual need. Fully automated; end-to-end – meaning that we can scale quickly and easily and ensure that Twill is future proof.

Thanks for reading. If you have any questions or if you want to read a blog about a specific topic, please contact me.

-Menno Veen –  Twill Growth Catalyst

What can Blockchain technology do for Twill?
Twill launches Supplier Initiated Booking

Related Articles