APS 'devoted' to user-centred design but not quite there with agile

User-centred design is becoming popular in the Australian Public Service, to the extent it is possible within the constraints of working in government, but it’s early days and strong evidence of the new approach’s impact is yet to emerge.

A panel of public servants working in service design, who discussed UCD at the most recent Canberra Evaluation Forum event, were nonetheless optimistic that it can lead to better policy outcomes, mainly by improving the ability to evaluate them.

Digital Transformation Agency chief operating officer Peter Alexander said he expected improved outcomes would be anecdotally reported at first. “Measurable improvement will come in time,” he added, as agencies get better at measuring and reporting service metrics, which the DTA is encouraging through the performance dashboards that are still a work in progress.

He said one metric in particular — cost per transaction — was a particularly important but challenging to measure. It is currently only available for two of the seven dashboards.

“That horrifies lots of public service agencies because it’s just something we don’t measure,” said Alexander. “And people will say it’s hard to measure, and, ‘I don’t understand how we’re going to apply it.’

“That metric set is really important and cost is really important, so when you consider, should you outsource a function, should you have another agency provide it … you need to understand that.”

He noted that any evaluation needs good baseline data about current policy outcomes as a starting point.

“I’d say we’re still a bit immature in that space of measurement — knowing if we’re doing any better and getting better outcomes for users,” said Alexander.

“But what I will say is we’re absolutely seeing a dedication — maybe even ‘devotion’ is a better word — from agencies to thinking about user centred design in services that are going out.”

Alexander was much more certain that UCD — which involves continuous improvement through ethnographic research that considers user needs and wants critically and independently of what they explicitly say — can at least provide better data for policy evaluation.

“I think it’s absolutely the case that good service design gets you better analytics, better evidence, and better policy development,” he said.

Department of Finance service design director Vanessa Roarty said some improvement in “completion rates and satisfaction” had come from applying UCD.

“We’re also seeing project benefits as well, so things like a reduction in risks and issues, more engaged stakeholders,” she added. “So if you’re actually using co-design practices — that’s designing with your users and with your stakeholders — we see those stakeholders more engaged, and we also see better decision making.”

Roarty told the public administration forum it was important to start ethnographic user research as early as possible, because there’s nothing more powerful than the evidence it provides to inform service design decisions.

“I think we still see a lot of hesitancy across government to engage with users in the really early stages,” she said. Not starting early enough is one of the most common problems with service design she comes across.

“When you’re conceiving the very first policy idea, when you’re sitting down and thrashing out how a service could work, that’s when you need to go and talk to users and really have a good understanding of the user need at that point,” said Roarty.

“Too often we have a service that we’re developing, we go and run some usability testing over it [and find issues] — we had one the other day and we found that one of the core concepts that the information architecture of the service was built on, users just didn’t understand it.

“They just didn’t get it and they were never going to understand it; it wasn’t something that was even on their radar, and having to actually go back and fix those things, as opposed to understanding their context and their level of understanding right from the start, is a very different thing.”

Talking to one user is better than zero

With ministers holding the decision-making power, the constraints of working in government mean life’s not so easy for the policy people, either. The fundamental structure of government that theoretically enables separate ministerial responsibility also creates the much-maligned public service silos that hold back collaboration.

It is a fact of life, as the panel acknowledged, that very often public servants won’t be able to do as much early user research as they would like, and as the DTA encourages through the Digital Service Standard.

“User-centred design is a always going to be a matter of scale, so it’s better to talk to one user than no users; it’s better to talk to 10 than to one; it’s better to talk to 100 than 10,” said Roarty.

“So it’s always a matter of scaling to the time and effort you have available.”

She also advises not to let the perfect be the enemy of the good when it comes to diversity among research participants.

“A lot of the time we’re just working with what we have available to us. You obviously want users who are as representative as possible but again, it’s better to talk to the users you can, than no users at all.”

Alexander suggested that in some cases, even vox pops with anyone in the street can be useful, depending on the question you’re trying to answer.

Roarty added that a truly ethnographic approach to research will provide some “insight” into user needs the users themselves may not have recognised. This is not a new concept; service providers like motor mechanics or dentists need to work out what their clients need because the clients usually have no idea at all.

The value of this kind of research also applies in cases where most citizens would not recognise the online channel as a ‘service’ — like with Centrelink’s online compliance initiative or a lot of what the Tax Office does.

Dealing with the ‘dark side’ of digital transformation

Consultant Pamela Kinnear threw a curly one at the panel, asking how user-centred design can help public servants address concern in the community about “the dark side of data analytics” and reassure people who distrust the agency and what it is trying to achieve.

The calls to boycott the last Census and the #notmydebt campaign against Centrelink were the examples she had in mind.

Roarty ventured that UCD would allow public servants to “dig a little bit deeper” into what is behind the distrust and suspicion.

“And then you can address the root causes of why people might not be bought in, why people might not trust government, and also you can understand the level and nature of that distrust and actually look at what treatments you can put in place to give a better experience to those people,” she said.

“Don’t ever be afraid of asking, even if you know you’re going to get a bad response.

“There’s a lot of government policy that’s ‘bad news’ policy, you know, so that’s no reason not to go and talk to users about it. We want to understand, how can we make this better, how can we make your life easier, how can we alleviate some of the stress that this might bring into your life.”

Alexander suggested the roots of both campaigns were more in the policy than the digital service design, and if the Department of Human Services had done more user research early in the piece, a lot of the issues that caused so much opprobrium would have come up.

“One of the core principles of user-centred design is around empathy for your users, and I think maybe that was something that was a little underestimated, … that emotional reaction people would have to what happened to them,” Roarty added.

‘Wagile’ methodology the reality of APS IT project delivery

The new service design standards for the digital era are clearly having some effects but obviously, the Westminster system places limits on the extent that government can change the way it works, and how quickly.

The bureaucracy obviously can’t suddenly switch to agile methodology and working in multi-disciplinary teams overnight, but public servants are making a concerted effort to apply the new logic of innovation and digital transformation however they can.

Alexander told one DHS employee at the forum that in reality, the current state of affairs was actually “wagile”– a transitional combination of the old large-scale IT delivery methodology and the iterative, user-centred approach that is taking over.

He told her that “we’ve basically got waterfall and agile and mashed them together, and tried to get the best of both, but what we’re really doing is taking the teams that were just doing our waterfall projects and telling them to deliver in sprints and do things faster,” adding that bar a few examples, agencies are not working in “true multi-disciplinary teams” yet, either.

“So, the perfect multi-disciplinary team has everyone you need in it, representing everything you’re going to need, the right technical capabilities to build the services you need, and it has a leader who can call on other resources when they’re needed to come into the team, or put people out of the team if they’re not needed anymore,” said the DTA’s COO.

However, he added that “what we’ve tended to see is teams that are incapable of doing that” because they are still unsure about how an agile team operates best or how they can do that within the APS context.

Alexander added that some foreign governments might do a good job of making it look like they are streaking ahead with service design improvements, but in his view the reality is they all face similar challenges and Australia is up there with the best.

He thinks it is more useful to try to match the quality of services provided by the private sector — noting that their CEOs have a lot more power than departmental heads in the public service. A lot of the barriers for bureaucrats are created by legislation, rules, regulations and cultural attitudes like those that killed the Australia Card and have loomed over other national identity projects ever since.

“Some of the things we’re doing around platforms, I think, will break some of those legislative and policy silos that agencies run,” Alexander added.

And it won’t just be the DTA delivering the new platforms. In line with its new approach of going into agencies instead of them coming to the digital agency for assistance, he said big organisations like the Tax Office, DHS and the Department of Immigration and Border Protection would be building them too.