Steve Waddington's Personal Thoughts on Exetel

I have thought about the various vendor and keynote messages delivered at Interop for a little while now, and the information has settled its way through various filters to a point where I can start to pick the signal from the noise.

Overall, the main thing I saw was how far three key technologies have come in the last 12 months. I concluded that is was a matter of necessity that caused it. Last year at the depth of the GFC Interop, and Las Vegas, was a sad place indeed. So much so, that the promise of technology efficiencies have been called upon, and demanded to be achieved.

The result of last years budget slashes and capex suspensions caused an acceleration in the deployment of two replacement technologies that could provide the immediate savings required from recast corporate budgets.

The outcome is three things that are important to the degree they will set the context for the next five years, possibly longer, of global IT strategies. They are:

1. Cloud computing
2. Visualization
3. Unified Communications

One point to note is that sometime around four years ago, I recall listening to John Chambers keynote at Interop where he said something along the lines if 'the network is the business'. While I understood that 'the network' was certainly needed to run 'the business', and good communications for both data and voice was vital for any business operations, it is on this trip that I can really see what was meant. A network, fully enabling the three points above IS the business, and the business is the network.

Here is why.

Cloud computing, pick your own definition, but in practical terms, is perhaps the biggest potential cost saver for any corporate IT department today. Think of the cost of, say, a departmental email server, the backup server for that, the DR plan and data archiving. Somewhere in the order of $20,000 over three years I would imagine, or maybe more. Compare that to the sort of pricing I saw at cloud computing vendor stands, which for that type of solution were in the order of $15-$25 per month. It is the sort of compelling cost saving that no IT manager, CFO or Board of Directors can ignore.

Hand in hand with the 'cloud' goes visualization. Without the cloud, it has great potential in its own right, but let's face it, it has been around for tend

I have this analogy that managing people is like standing on the back of a moving truck trying to hold on to several sheets of lightly oiled glass. No matter how good a grip you might have, it never seems enough, and one good bump and any one of those fragile things can slip and shatter.

Every occupation has its 'tools of the trade'. Carpentry can't take place without a hammer, nor plumbing without a stilson. Yet, it seems to me that in many cases someone is simply 'made' a manager by the bestowing of that title, and left to cope bare handed with the situation.

If there is such a person as a 'natural' manager, I have never met them. Just like a runner can't naturally win a gold medal, it requires a lot of commitment, planning and support to get to the level where a win is possible.

It makes me angry every time I see the NBN ads on TV; the implications the average 'uninformed' view is drawn to conclude about how great and fantastic the NBN will be. Good job by the advertising agency to fulfill its brief, disgraceful job by a government to give such a brief.

I was asked to comment on the coalitions NBN alternative the other day. Part of which appeared here.

Here is the full response:

Really, I despair. But ok, here is my view on the main points:

1. Deliver 25Mbps by the end of the first term.

Why is it stupid:

Right now, ADSL2 delivers up to 24Mbps. So the promise is a 1Mbps increase, for only 20 Billion dollars. But without spending 1 cent, the current broadband service providers would have, if the NBN had not killed off all investment, reached that mark in the not too distant future anyway.

Why it is better than the current plan:

Spending $20 Billion is better than Labors plan to spend $40 Billion+++, for still, essentially what we already have.

2. Fibre to the Node vs Fibre to the House

The statement "unlike the government’s fibre-to-the-premises technology, the coalition’s network would not be easily upgradable." is wrong.

The fact is, copper is a cheaper technology to deploy for short distance signaling, and even has advantages over fibre for ease and flexibility of installation. It is also easier and cheaper to replace a copper based network switch than to replace or upgrade a fibre switch.

Ask yourself, is your desktop or office server connected by fibre? No. It is connected by copper wire. To say that high speed data mandates fibre, at least for short distances, simply ignores physics, as well as the evidence in front of your eyes.

So fibre to the house, much reducing the roll out cost, is a point in favor I would say, if it weren't for the pointlessness of it all re: points 1 and 3.

3. Completed in 2019

Since it is 96% completed now - 24Mbps on ADSL2 vs 25Mbps, it is hard to see how that promise couldn't be kept.

4. Sell the network once it is ready

Well, we, taxpayers, already paid for the first network - PMG/Telecom/Telstra, which has been partly sold. Now we are paying for a second network, which will be sold. So I guess we can all look forward to a reimbursement check from the ATO once the sale goes through? Excuse my cynicism.

5. "opposition claims the government’s NBN plan could blowout to $90 billion – a claim the government venomously denies.'

And well they may deny it. $150bn is a more likely figure.

6. 25/50/100Mbps

In this blog post in 2009 I calculated the cost of a 1Gbps home user broadband service to be $8bn, so even allowing for inflation, $20bn still seems to high by a factor of 2:

http://steve.blogs.exetel.com.au/index.php?/archives/227-1Gbps-to-Every-Home-in-Australia...html

Unlike me, Google can put some serious money where their mouth is, and deliver a 1Gbps service:

http://news.cnet.com/8301-1023_3-57481108-93/google-shows-isps-how-to-build-a-superfast-network/

If it is inevitable that we have to spend the money on a new communications infrastructure, then 1Gbps is where we should be. That is what the technology of today can give us. So why does everyone want to spend so much just to end up in the same ballpark we are now?

7. “We will deliver a better NBN.”

The criteria for a 'better NBN' are:

- 1Gbps to the home
- at the same cost to the user as ADSL is now
- with a declining user cost over time as technology becomes cheaper, down to $20 per month after 5 years

Now that would really be something significant for the country and truly deliver the promise of an NBN.

... to deliver 1Gbps to the home in this CNET article

pretty much what I said 3 years ago in a blog post.

But good on you Google, for showing telcos how, I think your view has a much better chance of being recognized than mine.

What is a residential ADSL customer worth? I have heard a few people now mention the 'NBN land grab', where ISP's like Optus, TPG, iiNet etc want to grab as much of the current ADSL market as they can, because on the 'level playing field' of the NBN landscape, where everyone's infrastructure costs are equal, the ONLY differentiator will be retail price, and more customers means better amortization of business overheads.

To understand the value of a customer to a Telco, the place to start is to figure the revenue the Telco will be looking to make - not now - but in the 5 year future, assuming the full realization of the NBN roll out as it stands.

There are three services:

- Telephony
- Internet
- Entertainment; a very broad brush that covers online games, but in terms of consumer dollar means essentially video/TV on demand. A replacement for cable in other words.

Looking at average monthly spends per customer now we get:

Telephony........$30
Internet...........$40
Entertainment...$90

So $160 per month ARPU (Average revenue per user) for an ISP that can deliver those three services via NBN.

If we assume a margin after delivery and infrastructure costs of, say 30%, give us close to $50 per month, or over a 2 year contract, $1,200.

But why would an ISP expect someone to immediately churn at the end of the contract? The delivery infrastructure is the same, the costs are the same. Why does anyone buy BP rather than Caltex? Only marketing and price as far as I can see. All things being equal, once connected to an ISP of your choice on the NBN, unless there is some compelling reason to change, you are going to stay with that ISP. So maybe the average 'life' of a customer will be 4, 5 or even more years.

Here is the current poll on Slashdot that gives some validity to that guess. The poll responders being mostly in the 'expert user' category, and mostly American.



Telco's, differentiated from non-Telco ISP's, tend to think in 15 year cycles. On that basis, 2 years can be pretty much written off as simply marketing and acquisition cost. Which puts the 'value' to a Telco of a residential customer, that they don't currently have, at $1,200. At least by my back of a napkin, rough estimate, estimate.

But then think about this; I was told, by a reliable industry source some time ago, that the Big Pond 'win back' campaign of 2 years ago had a reserve of $900 discretionary allocation they could use to entice customers back to their network. Also, I have heard from several other people, who I have no reason to doubt, that ISP's with their own DSLAMs, like iiNet and TPG, see the value of acquiring a customer at about $900. And that information is now at least a year old, so maybe my $1,200 isn't too far off the mark.

Maybe those guys figured out long ago what a residential ADSL customer is worth. I am feeling a little slow today.

In 2009 we implemented a customer survey program for support issues, when a ticket was closed. It was done for the obvious reasons; to get the feedback direct from the customer on how the case was handled; get the information direct from the source on what we may need to improve on; and track changes/improvements we make over time to make sure they are effective.

The results, graphed over 2 years, showed a steady month on month improvement, and this tracked nicely also with a general decrease in emails to 'complaints@exetel.com.au' and TIO case reduction. Because each survey response is linked to the case ticket, we can see who has worked on the case, and the support supervisor or manager can review any low scores and identify why we didn't make the customer happy.

In our own terminology, we dubbed a high score response - where the average is 4.5 - a 'bingo', and a low score response - where the average is less than 3, a 'bongo'. Bingo's were encouraged, with engineers who achieved consistent high scores being recognized for their work in their quarterly reviews. Bongo's on the other hand were treated very seriously, triggering a 'Negative Customer Feedback' email to be sent to support supervisors, manager and the Directors, and requiring an cause analysis and corrective action to be undertaken.

The directly measured improvement over time, along with improvement in related metrics, showed the program was successful. But still there seemed to be something missing.

By giving such focus to the low score responses the focus was always on the negative, rather than the positive aspects of what was being achieved. And while support engineers were recognized for achieving good score, the recognition was well after the event.

So in February we began the 'Bingo Bonus' scheme, where the support engineer gets a pay bonus for each 'bingo' feedback response they get from a customer; qualified by three simple rules:

- if two engineers are involved in the case, the bonus is split 50/50
- if more than two engineers are involved in the case, that case is not eligible for a bonus
- any bongo response disqualifies that engineer from any bonus for that month

The results speak for themselves. In the December 2011 there were 35 Bingos and 5 Bongos, the average handling score was 1.2 (note on what this means at the end of this post) and the reputation score average was -0.3.

January was similar.

The scheme started mid February, and the early results hadn't changed much - 35 Bingos, 5 Bongos, handling score of 1.16 and an improved reputation score of -0.1

But by May, Bingos had increased to 56, Bongos was still 5, handling score was up to 1.58, and reputation had turned positive to 0.1. Then in June: 75 Bingos, 4 Bongos, handling 1.59 and reputation 0.5. Since we are dealing with averages of hundreds of results, that sort of increase represents and order of magnitude improvement.

The total bingo bonus monthly payout for June is about equal to the salary of 1 level 1 engineer. So to look at it another way, would the addition of 1 more level one engineer equate to an equal improvement in customer satisfaction? Highly unlikely. And it certainly wouldn't have given the other engineers the motivation and incentive to achieve as they have.

Note on what the survey scores mean:

Reputation: Q1 asks what the customer how they rated Exetel before they has the problem, Q7 asks what they rated us after. The reputation score is the difference between the two, averaged over all responses.

Handling: Q2-6 ask the customer to rate aspects of how the case was handled, the result is averaged for each case, subtracted by 3, and then averaged for all responses. We subtract 3 from the score so that when the results are graphed, the reputation line and the handling line are both in the same scale, and it is easy to see at a glance where the handling line drops below zero. A handling score of 1.5 equals a 4.5 response score average from the customer.

Well, we ended the year with a bang, both good and bad.

The good parts were:

- Another record sales month for corporate services with over 200 new services signed for the month
- Outbound sales in Colombo firing up, sales targets well on track there and making a significant contribution to our sales figures
- 6 successive months of downward trends for customer complaints
- 6 successive months of upward trends on customer satisfaction surveys
- 4 successive months of improved indicators across the business

Very gratifying to see these results, which certainly have not been achieved without a lot of hard work, some tough, sometimes unpopular, decisions and a few tears along the way.

And the bad part - a literal 'bang!' was the failure of AAPT's IP bandwidth on Saturday:



We are currently waiting for the official RFO, but from an engineering perspective, it is pretty clear to us what happened. It is also clear that AAPT are going to have to take a hard line within their own network engineering group, cut through the finger pointing and blame avoidance that is bound to happen, and fix themselves up pretty damn quick - if they want to retain our, or anyone else's business in the future.

P.S. Edit: Leap second injection has been offered as the root cause.

May set a new record for revenue, and new records in:

- Corporate sales orders, beating the previous sales record of 150

- Provisioned corporate orders, ending on 102

- SMS sales, exceeding 950,000 in a month for the first time

- VoIP minutes

- Residential new business revenue

While residential (ADSL) sales didn't quite match the record month of February, the new, higher value Anzac and Fair Use plans pushed revenues for new orders just over the February mark. All in all it was a great pick-up from April, and we are now at 106% YTD sales target for corporate sales and just a shade over 100% YTD forecast for residential sales.

June, as usual for that month, is shaping up to be another record killer. Unless I have just jinxed it.

The R&D program we began with the Sri Lanka Institute of Information Technology two years ago is in the final stages of completion. Yesterday we held a press conference, organized by the SLIIT, to announce the results of the project. The press coverage included, all the national TV stations, papers and the local Al Jazera new team.

The presentation I gave follows, but the most impressive part of the conference was Prof. Kumarawadu's demonstration where he was showing question/answer response illustrating the confidence level algorithms that form the core logic of the system. A few minutes into the demonstration of the web based interface, I noticed that the url was http://calliope.exetel.com.au/ - our live production system. Not some mock-up put together for the press, not some abstract diagrams of an ideal system, but the real, live, in production, result of the development work. Now if that isn't confidence in your work, then I don't know what is.







My presentation:

Distinguished guests, professors, members of the press, Ladies and Gentlemen,

I am the CEO of Exetel and we are a small Australian company and have been operating in Colombo for four years.

From the start we saw the potential for collaboration with academic institutions in Sri Lanka.

There are many difficulties that any company must face starting an operation in a different country. One of the hardest things we found with regard to our academic collaboration program was how to engage in the first place.

I am not sure why it was, but for the first six months when I was trying to establish this program, I could get no traction at all from any of your universities. You would think that the offer of fully funded scholarships and research funding would have some appeal, but it didn't seem that way.

Maybe it is a reflection of the excellent education system and government funding already in place, that private enterprise funding is not needed or wanted. As I said, I don't know.

But, what I can say is that we did overcome that hurdle, and the SLIIT had the initiative to engage with us in, what was to become, three very successful projects.

Before I talk about the project, our AI engine called Calliope, I would like to first speak a little about, what I think, is at least equally beneficial, and I feel a lot of pride for.

The first initiative we undertook with the SLIIT was to provide two fully funded scholarships per year, in the field of Unified Communications. We started that program in 2009, and we now have six SLIIT students receiving 70,000 rupee per semester fully funded scholarships.

But what I am particularly proud of in that project is that the scholarships are completely without encumbrance. That is; we ask for nothing from the students, other than that they maintain a GPA of 3.0. We do offer the opportunity to paid internship work, but it is not mandatory. On graduation, our intention is to offer the students full time work at Exetel. But this also is not a mandatory condition.

Our objective in doing this was not completely altruistic. We wanted to gain access to the 'best and brightest' graduates, who would see our company as the best place to work for them, to begin and their working careers.

We were not looking for indentured servitude. We wanted bright, thinking, capable people, who wanted to work for us. And what I am very pleased to be able to tell you today, is that is just what we have achieved.

We currently employ 13 SLIIT graduates, paying them the highest salaries of any company in Sri Lanka, and providing technical support and network engineering services to our Australian operation that is the equal of any in the world.

Our second initiative has been to offer paid internships to final year students. We have always had great success with this program in Australia, where students have worked as interns, gained full time employment on graduation, and gone on within Exetel to become team leaders, supervisors and managers. In Sri Lanka, three people who started as interns are no full time staff, and we have two current interns.

We have made those investments in good faith. And it is a tribute to the quality of the SLIIT graduates that our faith has been vindicated.

The third initiative is the reason we are here today. The culmination of the funding of of the SLIIT research project into Artificial Intelligence, which we call Calliope.

Calliope is from the ancient Greek, and was the muse of epic poetry.

And bringing this project to fruition has been something of an epic.

So what is Calliope? It is simply this; a system by which a customer use natural English to find out the information they want. One way of doing that is for a customer to call a call centre. And for complex, difficult problems, that is exactly what they should do.

But so many, by far the vast majority of calls to any call centre are simple questions, usually answered somewhere on a company web site or FAQ page. But for the customer, it is far more convenient to just call someone than to try to find the information on a web site, no matter how accessible the company thinks it may have made that information.

So eight years ago we conceived of Calliope as a method to make us a more efficient company, and make the information our customers wanted more available to them.

Our own development stalled in its second year. We had simply reached the limit of the resources a small company like us could commit to such a project. And the concept languished for a few years.

We were not by any means the only company to consider such an approach. There are a number of such AI's in commercial use. But from our point of view they all had the same failings: they were expensive to by, hard to maintain, and as far as we could see, only able to deliver a valid answer about 50% of the time.

We wanted something better than that.

You may have heard of the recent current 'killer app' for tablets and iPhones called 'Siri'. On an individual basis, Siri is exactly the sort of intelligent agent that Calliope is on a commercial basis. You could say Siri is to Calliope what a spreadsheet is to Oracle database.

Two years ago Exetel began a collaboration with the SLIIT through funding to develop an AI system that was commercially viable. Exetel had three criteria that needed to be met to achieve this;

First the system had to draw from a single source of information - the Exetel Wiki site.

Second, it needed a knowledge base that was easy to maintain. A single source of update information, which was our own company knowledge base – our wiki site, would achieve that

And third it had to integrate fully into our existing customer management systems.

The SLIIT development team have met those objectives to the extent that the systems is and integrated and valuable part for our customer support.

The benefits Exetel and its customers gains from this system are:

- Skilled support engineer time is freed from mundane repetitive tasks that the AI can handle, and can concentrate on more difficult solutions

- Customers can get immediate responses to all basic questions about support and provisioning, and have direct access to the information they need

- Exetel has reduced its cost for basic support, and increased the quality of support to customers

The successful culmination of this project is one more demonstration that Sri Lanka has much more to offer the world than Tea, gems and garments. For a small company like Exetel to be able to engage and successfully complete such an endeavor with the SLIIT stands as an example of the intellectual investment value Sri Lanka can offer.

And so in closing I offer my congratulations and sincere thanks to Professor Kumarawadu and his development team, and the SLIIT for engaging in this project with us, and to make our vision and initiative a success.

Thank you.

We extended the offer for Fair Use plans through April, and will most likely keep them available for May as well. They have proved pretty popular, though, as the only 'no contract' plan on the market, and better value over 12 months than any 'unlimited*' plan, sales have not been has high as I would have thought. Still, it was a good enough deal that I bought it myself for my family use - so no more excess use surprises for me when my son at Uni downloads his semester video assignments in RAW format.

I guess I can understand why some people might be reluctant to buy a Fair Use plan - where the clearest definition of Fair Use we can use is based on an average of use. And if that were the only criteria, maybe we could take an historical average and publish that as an indication. Well, not really - so I am told by our complaisance officer - if we publish a figure we have to stick by it. The other thing is, the average use is quite low, less than 100GB overall, and that is just not an appealing number, where legitimate use nowadays can see in three or four times that downloaded in a month. So 'Fair Use' does require some two way trust; the ISP needs to 'trust' the user not to download the entire Internet, and the user needs to 'trust' the ISP not to penalize them for some arbitrarily low, albeit average, download amount.

Bottom line, no matter how good the deal, Fair Use plans are not for untrusting people. (and I am not using the term 'untrusting' as a criticism of anyone, maybe 'savvy consumer' or 'healthy cynicism' would be better terms)

So what to do?

We came up with the 'Anzac Special', no contract, 1TB allowance, $55pm - same as the Fair Use plan, but for people who want certainty in quota rather than certainty in excess usage charges.

The plan should be on the web site later today, and we will see how it runs to the end of April and decide if we want to extend it into May.

Nearly four years ago we put in place a system to forward on copyright infringement notices that identified an IP address to the account to which that IP address was assigned. The system has been running faithfully for, today, 1299 days, and sending a report each day. Graphing the results shows an interesting trend:



Considering the amount of bandwidth available to residential subscribers has almost quadrupled over that time, the possibilities are:

- Copyright infringement notices and anti-piracy campaigns are effective over time
- P2P stealth has got much better
- Users pirating have got pretty much all there is to get, and there isn't much left
- Copyright enforcement companies have become less active

I don't think the last point is true, their systems would be pretty much automated by now I would think, so there is no particular reason so think they have scaled back.

P2P stealth I don't think is the reason either. No matter how much encryption, both ends still need to know the IP address and the file name, and that can't be hidden transferring a file to/from a 'honey pot' peer.

So, is it Hurrah! for the good guys or has all the gold that can be plundered been plundered from those Spanish galleons?

We were talking about 'unlimited' plans on offer in the market, and how they all have one or more asterisk, tilda or some other super text that points to a 'fair use' policy that explains how 'unlimited' doesn't really mean unlimited. And why the ACCC. for some reason, requires 'unlimited*' actually be 'unlimited'.

So, why not, we mused, just call the plan 'Fair Use' up front - flat rate, no metering, no shaping, except in conditions where it is required - which is what the fair use policy says. Just instead of 'hiding' it and hoping no one notices the tricky conditions of 'unlimited*', have fair use up front, and explain what makes it unlimited. In fact, do away with any word as possibly misleading as 'unlimited' and state plainly what is being offered.

Maybe there is something we all missed in this, why no one does it? I guess we will find out.

The 'soft release' of our Fair Use $55 and Fair Use 65 plans will be on the web site later today. It will be a limited time offer, and we will see how it goes.

I see Delimiter have noticed our new NBN pricing. A fairly well balanced article, but we do have a high use plan which they didn't mention - for a 300Gbps quota. And, famous last words, 300Gb should be enough for anyone, right?

Another important difference is the Exetel quota is download only, in theory, that makes it double the quota of other ISP's who include uploads in the quota. Though in practice, it works out to 15-20% extra, or whatever the ratio of downloads to uploads for a particular user.

I don't know why it is that everyone reporting on the NBN, Delimiter's article included, seems to be using this early 'get it off the ground, feel out the market' pricing as proof of future NBN pricing. It can't have escaped everyones attention that introductory pricing is often much cheaper than the price something will be once it had gained market acceptance. I don't think the NBN will be any different.

Knowing the NBN cost, the number of people who will use it, and the way it will be paid for, it can't be too hard to work out what the price will be for access. Something that hasn't escaped the Coalition's notice, but does seem to escape journalists. Oh well.

We have been running a trial marketing program for a bundled SMB product for the last month. I really like the idea of this sort of product, it meets a 'niche' - but that is a very big niche, of over 1,000,000 businesses.

The marketing is expanding throughout February, and we will be moving support engineers into a specialist SMB support group in March. Which is hand-in-hand with the plan we started in November to increase the number of sales and support engineers for both residential and corporate - ahead of the 2012 sales forecast.

Well, the sales numbers are right on track (better than forecast for residential in fact), so our hiring plan has been right on time.

It is very nice when plans coincide with targets so well, even nicer when the timing works out so well too.