Electoral commissioner Tom Rogers is “incredibly proud” of his team’s efforts in conducting the 2016 double dissolution election and stands by the integrity of the count, despite shortcomings in procurement processes and loose risk management identified by the auditor-general’s office.
Rogers defends the statutory agency’s work in his published response and argues it was actually reasonably successful when the very difficult circumstances created by legislative changes and a tight timeframe are taken into account.
But there’s been little sympathy for the public servants who support the commission in commentary around the Australian National Audit Office report.
The audit examined procurement of secure transport for the ballot papers and an electronic vote-scanning system for the Senate, which was required because the tried-and-tested manual counting process would take too long, following changes to the preferential voting system.
“Insufficient emphasis was given by the AEC to open and effective competition in its procurement processes as a means of demonstrably achieving value for money,” the report states. “Its contract and risk management was also not consistently to an appropriate standard.”
The details behind these main findings provide plenty of fuel for unflattering analysis in the media, especially around non-compliance with cyber security controls for the scanning system, and reduced assurance of the Senate result’s integrity.
But while Rogers accepts the report’s recommendations, he told the ANAO its audit report was “potentially misleading to the reader” because it doesn’t focus much on the profound challenges the AEC faced due to the legislation passed by federal parliament, essentially to limit the number of candidates from tiny parties winning seats through preferences with very few primary votes.
He argues that “any logical review of program delivery” needs this kind of context and suggests that with a fuller account of the challenges faced by the AEC in dealing with the biggest voting reforms in 30 years and a few other complications, the “extremely useful observations” of the auditors would be significantly more valuable:
“In the extraordinarily short period of three months, and without prior warning, the AEC successfully developed and then implemented a robust, effective, technologically advanced and entirely new system for counting, under high levels of scrutiny, some 15,000,000 Senate votes in multiple locations around Australia.
“Further layers of complexity were added by: predictions of a close event (with attendant media and political focus); the election being a double dissolution; the election period following the very recent finalisation of several major boundary redistributions; a shorter than usual timeframe specified for the return of the Writs; the need to develop, test, and deliver a nuanced national education campaign for all voters about the changes; and the election being the first national event since the implementation of the Keelty Report recommendations following the 2013 federal election.
“Notwithstanding these additional complications, the AEC was keenly aware that failed delivery, non-delivery, or even partial delivery, of the Senate voting reforms would have had catastrophic consequences for Australia’s system of governance with both domestic and international implications.”
Procurement problems
It seems the rush to prepare for the election led the AEC to pull out some of the stops in procurement, leading the auditors to admonish it for not using the power of market competition enough. Although Rogers seems happy that the agency got the job done without a major catastrophe, the ANAO suggests the AEC could have achieved better outcomes for a lower price:
“The AEC has not demonstrably achieved value for money in its procurement of Senate scanning services. It has not used competitive pressure to drive value nor given due consideration to cost in its procurement decision-making.
“The AEC sought to encourage competition amongst transport providers but at times struggled to achieve value for money. It would have benefited from additional logistics expertise and transport industry knowledge when establishing and managing transport arrangements.”
There was some consideration of costs versus benefits in the transport contracting, though not enough evidence of value for the ANAO’s liking. With the ballot-scanning system, which was provided by Fuji Xerox Businessforce, it seems the price was of little concern.
“No consideration of financial cost was evident in the records of the AEC’s decision-making to implement the Senate scanning system.
“Timeliness, quality and risk were taken into account. The documentation on the Senate scanning system procurement indicates that inadequate consideration was given to assessing value for money and did not demonstrate that it was achieved.”
The audit office’s “key learnings” for the public service include the importance of open tender processes for significant procurements to make sure agencies can choose from the widest possible range of suppliers to find the best solutions at the best price. The report adds a note on panels:
“A decision to use a panel established by another entity should be informed by an assessment of how long ago the panel was established, whether the other entity’s approach to the market (and the resulting contractual arrangements) clearly provided for broader use of the panel, and the extent to which the goods and/or services are covered by the panel arrangements (including prices having already been established).”
The ANAO also reminds agencies they need “a full understanding of what they are buying, and of the market” to make optimal procurement decisions.
Scaled-down cyber security
The audit found the AEC also took on more IT security risk than it would normally, under pressure to rapidly procure and implement the electronic scanning system. As a result, it could not demonstrate enough assurance of the integrity of that section of the vote, the report warns.
“Insufficient attention was paid to ensuring the AEC could identify whether the system had been compromised.
“The Senate scanning and transport suppliers delivered the services as contracted. The AEC had limited insight into whether its contractual and procedural risk treatments were effective. Going forward, the AEC needs to be better able to verify and demonstrate the integrity of its electoral data.”
Rogers stands by the overall success of the system (as does the supplier in its own response letter) and he points out it even won two awards in the ACT round of the Australian Information Industry Association’s iAwards.
“On any reasonable measure the solution was an impressive accomplishment which functioned as intended,” he told the audit office.
While there were some issues with security of the physical ballots — explicit provisions that they never be left unattended were omitted from seven of 17 transport contracts, and there was a failure to check the “political neutrality” of the logistics workers — the report raises a longer list of concerns about data security and assurance of the computer-assisted results.
The only real way to provide scrutineers with meaningful evidence of the accuracy of the outcome is to have a public audit of the paper evidence against the published preferences.
— Vanessa Teague (@VTeagueAus) January 23, 2018
The agency has been criticised in news reports by a range of IT, cryptography and information security experts, such as Dr Vanessa Teague, a University of Melbourne researcher who is particularly concerned about managing the risks of digitised electoral systems (and the separate issue of open data re-identification).
A group of academics including Teague, who has argued the AEC cannot prove the reliability of the results, sent the agency a paper proposing four different methods for a statistically valid audit of the Senate votes in the days after the election, including open source code they could apparently use. The ANAO report notes the agency has not done such a statistical audit, and that cyber security experts inside government had identified the lack of a plan for one as a significant risk.
“That is, without audit and analysis, the AEC may be unable to identify with any level of confidence whether the system had been compromised and whether a recount was required.”
News reports have focused on a decision to increase the level of human oversight in the semi-automated Senate counting process (pictured above), at additional cost of up to $8.6 million. As ANAO puts it:
“To improve integrity, a late decision was made for all voter preferences to be entered by a human operator in addition to being captured by the technology. Any mismatches between the human’s and the technology’s interpretation were investigated and resolved. The AEC does not know the number or nature of mismatches to determine if this was a cost-effective risk treatment.”
According to Rogers, the point was to reassure politicians and other “external stakeholders” that they could trust the Senate results.
“This decision was not taken because I doubted the integrity of the data enabled by the solution design, but because I felt it necessary to maintain stakeholder confidence in the outcomes of the process – the first time that Senate data would be entered in that manner,” he told the ANAO.
The AEC’s contract with Fuji Xerox Businessforce did not require compliance with the federal cyber security framework, and the agency found that a quarter of the relevant security controls were not implemented.
“The security risk situation was accepted by the AEC but was not made sufficiently transparent,” according to the report, although it acknowledges that real-time security monitoring was enough to conclude there was no large-scale deliberate vote tampering.
A 10-month delay before asking the IT company to delete the voting data from its system has also added to concerns.
Rogers remains “highly confident” of the integrity of the data produced by the process followed, and also rejects any claim that he was misled by his staff about compliance with the federal Information Security Manual.
The audit found that, at first, deputy commissioner Kathryn Toohey reported the system was fully ISM-compliant before amending that advice to say it wasn’t correct, 10 days later. “I asked the team to conduct a risk assessment on this,” she told Rogers, and said this found there was only one low-level risk and one medium-level risk.
“Given the tight timeframe the team have done everything reasonable to manage the IT security risks,” she added. “Importantly … we addressed all 8 ASD IT security recommendations.”
The ANAO disagreed:
“The two risks referred to in the annotation did not encompass the risks associated with being non-compliant with 107 in-scope ISM controls. Further, the ASD had made 19 recommendations and not eight as advised to the Electoral Commissioner. The ANAO’s analysis was that the documented advice to the Electoral Commissioner did not fulfil the ‘need to ensure the agency head has appropriate oversight of the security risks being accepted on behalf of the agency’ stipulated in the ISM.”
The audit found an external transparency issue, too. Essentially, the AEC decided to accept a relatively high level of cyber security risk due to the short timeframe, but published materials that obscured the choices it made from readers and made it sound like everything was hunky-dory.
There’s a lot more in the report on the loose approach taken to risk management due to the short timeframe and complex task for the AEC, and while Rogers stands by the decisions that were made and the resulting outcome, he has also promised to learn from the recommendations.
He defends the transparency of the actual counting process on the day, and notes again that there were already various politicians and interested experts watching closely throughout the rushed process to deliver on the significant electoral reforms.
“The AEC is aware that there are a range of experts and academics who have made observations, both positive and negative, on the solution implemented,” said Rogers. “The AEC remains confident that the range of measures put in place for the 2016 federal election ensured the integrity of the Senate count.
“Indeed, the ANAO report does not cite any evidence to the contrary. However, for future events, the AEC will continue to evaluate and if appropriate, implement additional assurance mechanisms to maintain the integrity of the count.”
In the “key learnings” on governance and risk, the audit report urges agencies to consider the costs and risks of automation — although the AEC had little choice in this case — and warns against set-and-forget risk management. The “risk treatments” that are put in place for new technology need to be monitored.
Image: Australian Electoral Commission.
Any feedback or news tips? Here’s where to contact the relevant team.