Original article Total Telecom:Read More
Interview
We caught up with co-founder, CTO and CEO of ITcare, Andrian Visnevschi, to discuss scaling operations and the elusive nature of building autonomous networks
The UK’s fibre sector has a diverse and increasingly complex approach to network operations management. The largest fibre network players can afford to run 24/7 network operations centres (NOCs), manned by specialists to provide round-the-clock monitoring, fault management, and service assurance.
For the smaller players, however, the cost of a 24/7 NOC is often prohibitive, making alternative operational models more attractive. In some cases, this means network oversight by junior staff, or simply an automated alert sent to an on-call engineer, neither of which guarantees a fast response.
As Andrian Visnevschi, co-founder, CTO and CEO of ITcare, explained to us, this can result in costly delays.
“A lot of operators today simply don’t have 24/7 coverage. In some cases, senior NOC engineers are working a typical 9–5. If something happens outside of business hours, in the best case a more junior colleague calls them. Often, it’s just an automated notification from PagerDuty or a similar tool, which can easily be missed or ignored,” said Andrian. “Then the engineer has to wake up, or get home, or pull over the car, acknowledge the alert, find a stable internet connection, connect to the network, and only then start troubleshooting. That’s not a fast reaction. In fact, the mean time to repair of those issues is rising and the customers are suffering.”
Delays in fault repair cause significant damage to customer relationships and undermine the commercial value of the network – both factors of crucial importance in the UK’s highly competitive environment.
“A lot of operators are struggling to survive in the new race for service quality,” said Andrian, highlighting a recent market shift from network coverage to service take-up and reliability. “There is huge pressure to provide quality services without operational costs rising sky high.”
This is where ITcare’s model differs. Rather than a service desk that watches dashboards and escalates, ITcare’s NOC is staffed around the clock by senior network engineers who work across multi-vendor environments, from fibre ISPs and wireless operators to data centre fabrics, and resolve incidents themselves.
“A real NOC isn’t a service desk. The engineers on shift can log into the device, find the problem and fix it at 3am, without waking anyone up,” said Andrian. “They treat our customers’ networks as if they were their own. Every NOC customer we have signed is still with us today.”
HORA: The whole network in one view
For ITcare, which runs managed NOC and network engineering services for telecoms customers, the first step to improvement is a unification of systems and data. Inventory mistakes, poor data visibility, and unclear network topology are commonplace, making resolving issues a slow, complex process. This was one of the primary reasons for the creation of HORA, ITcare’s AI network operations platform, which gives NOC engineers full visibility of the network in one place: inventory and source of truth, metrics, logs, traffic flows, BGP monitoring, live network state, configuration backups and controlled change management, with AI correlating across all of it. ITcare built it for its own NOC first, and its engineers use it every day on live customer networks.
“By combining all of the available data, HORA can quickly troubleshoot and provide root cause analysis. It can also show exactly which customers are affected by a specific outage,” said Andrian. “With all the network data in one place, engineers stop jumping between five different tools. That makes them far more efficient, lets us automate more of the routine work, and cuts response times.”
HORA is designed to be slotted over the top of the network, being on-prem by design and network management system-agnostic. All networks are different – Andrian talks of “fine tuning” HORA to each customer – but fundamentally there are many NOC processes that are similar, regardless of tech stack.
“Over the years, we’ve worked with different ISPs, cloud hosting providers, and data centres worldwide, and our engineers are often doing the same repetitive tasks with each. They may use different hardware, different network architecture, and different protocols, but the troubleshooting chain they all use on a daily basis is usually quite similar,” said Andrian. “That’s where AI can be useful. We’re able to automate some of these tasks to make engineers’ lives easier.”
In ITcare’s own NOC, HORA has cut median investigation time from around 28 minutes to under four, roughly seven times faster than manual root cause analysis.
HORA’s change management capabilities also mean that changes are clearly documented.
“Engineers are usually pretty lazy,” Andrian joked. “They like making changes but they don’t like updating the documentation. HORA not only creates a change management process for you to approve but also handles the documentation so it is always up to date.”
AI guardrails, and Autonomous Level 4
Despite being bullish on the benefits of AI integration, Andrian is no AI evangelist, describing ITcare in a previous Total Telecom article as “conservative in a specific way”. HORA underwent extensive testing in the company’s own NOC before being offered to customers, with strict guardrails in place.
“We’ve all heard horror stories about AIs connecting to data bases, erasing all the data, and even generating random data to replace it,” he said. “You’ve got to be really careful when you build these tools. In HORA, the AI can propose a change, but an engineer reviews and approves it. Nothing touches a device without that approval.”
These safety concerns are, at least in part, why attempting to rapidly progress to Level 4 (L4) autonomy – in which an AI system monitors the network and handles tasks without human intervention – could be highly dangerous.
“I think [many in the industry] are all rushing it too much. It’s really so complicated to reach L4,” said Andrian. “If you’re building something solely for your own operations and infrastructure then it may be possible. But as soon as something changes in the design or a new vendor is added, you’ll need to start again.”
Keeping expert humans in the loop
For Andrian, the use of AI in NOC management is not about replacing engineers, but rather enhancing their abilities so networks can be scaled effectively without expanding overheads.
“It’s about allowing you to scale easily. With HORA, a single engineer has a tool that can quickly get to root cause, assess the impact, and prepare a change across hundreds of devices from a plain-language request, which the engineer then reviews and approves,” said Andrian.
“The tool doesn’t replace the engineer. Our customers get senior engineers on shift around the clock, and HORA makes each of them faster,” he added.
For operators under pressure to improve service quality while controlling costs, the immediate opportunity lies in pairing experienced engineers, whether in-house or through a partner, with tools that let them work smarter and scale operations more efficiently. Full autonomy remains a longer-term ambition.
Andrian Visnevschi is CEO and CTO of ITcare and holds JNCIE-SP #2975. ITcare runs managed NOC and network engineering for ISPs, data centre operators and hosting providers worldwide, and builds HORA, an AI network operations platform.
The post Smarter NOC management in the age of AI appeared first on Total Telecom.