Regarding the Freedom of Information “hack”

By evan on Apr 12, 2018

Update:

There is now a legal defence gofundme started by one of the CanSecWest organizers. Please donate what you can. This is a very important case, the government can’t be allowed to get away with this.

CBC has granted the teenager anonymity, but Jack Julian has a very good report on what happened from the teenagers point of view. 

/Update

Nova Scotia’s FOIPOP web service, much to the chagrin of reporters, has been unavailable for the better part of a week. Ironically, not much information has been provided on why. Today HRP and the Minister of Internal Affairs announced the web service had been “compromised” and a suspect was in custody. I’ll leave coverage of the subsequent political posturing to the news media, and instead focus on the actual attack and the implications this case has for security research in general.

The FOIPOP Webservice

Before I get into the details, I should explain what the provincial FOIPOP web service is and how it works.

It is a government-owned, subcontractor-run portal to pay for and receive FOIPOP reports. As a citizen, or as a reporter, you can pay $5 fee, and get access to government documents, from the normal course of their business, that are by and large considered to be public. In fact, there is a law, the Freedom of Information and Protection of Privacy Act that ensures those records are available to the public, with some restrictions. Those restrictions largely surround personal information. For example, I can request information about a project, but not about a person unless that person is me (or has given permission.)

Let’s get an idea of the scope of this breach. According to Global News,

On April 6, Unisys informed the province that between March 3 and March 5 more than 7,000 documents  were accessed and downloaded by a “non-authorized person.”

The province says that 250 of the documents contain highly sensitive personal information such as birth dates, addresses and social insurance numbers.

This implies there were 6750 documents that did not contain “highly sensitive” personal information and 250 that did.

As Tim Bousquet at the Halifax Examiner reported:

Part of my routine for writing Morning File is to daily check various government websites for new activity — provincial and federal tender offers, orders in council, and the Freedom of Information Office’s disclosure log.

That last is a bit of reporting theft — we reporters can see what each other has been working on, as the FOI office posts the disclosures given to other reporters two weeks after they’ve been released. More importantly, citizens can use the site to easily make their own Freedom of Information requests, pay the $5 application fee, track their requests, get an electronic record when the information is released, and like reporters do, look at other releases.

Considering 6750 of the documents did not contain “highly sensitive” personal information, and were therefore literally published publically by the government, that would imply to me that the actual scope of the breach is limited to 250 records.

The Attack

An unnamed 19-year-old man from Halifax (I’m calling him Mr. Big) was arrested, interrogated, and charged yesterday in relation to a “breach of a provincial government network” and was subsequently charged with “Unauthorized Use of Computer” which carries a penalty up to 10 years. As Deputy Minister Jeff Conrad told Global News

“There’s no question, this was not someone just playing around”

It would appear the government is not “playing around” either considering this charge carries the same maximum sentence as both rape, and creating child pornography.

We’ve established that 250 records were “highly sensitive,” the question is how did Mr. Big retrieve them? Surely the provincial government does it’s best to protect “highly sensitive” documents from hackers. Right?

The Exploit

I wish I could say the exploit was advanced. That it was complicated, that it was novel, or new; That the province simply had no chance against this bastion of elite hacker skills. The problem is I can’t even call it an exploit with a straight face. Ernie and Bert probably explain best.

The way the documents are stored is simple. They’re available at a specific URL, which David Fraser, a Halifax-based privacy lawyer, was happy to provide:

https://foipop.novascotia.ca/foia/views/_AttachmentDownload.jsp?attachmentRSN=1234

Document number 1235 is stored at https://foipop.novascotia.ca/foia/views/_AttachmentDownload.jsp?attachmentRSN=1235.

Guess where document 1236 is stored? This is not a new problem. In fact, it was recognized over a decade ago as one of the top ten issues affecting web application security. All Mr. Big had to do is add.

The software is manufactured by a company called CSDC Systems. As CBC reports;

“This is an isolated incident and no other CSDC products or customers have been impacted,”

I was able to find several American cities using the same software, and they all work the same. That would imply the system is working as designed. I believe them when they say the issue is isolated to NS because this is not an issue with the software but how it’s use by the province.

https://eservices.iowa.gov/PublicPortal/Iowa/IBON/common/display_attachment.jsp?AttachmentRSN=2908
https://lic.ok.gov/PublicPortal/OAB/common/display_attachment.jsp?AttachmentRSN=392874

These two sites are very interesting, because they use the same software, but are in a subfolder called “PublicPortal.” We’ll get back to that.

You can find them yourself, simply google “inurl:attachmentRSN”. Try it out, and you’ll notice the first few results are from none other than foipop.novascotia.ca.

I later found the same URL on the NS NDP website. The link doesn’t currently work as the province took the system down. That being said, Google was able to index and cache, several FOIPOP requests. This document specifically, number 7433, appears to have all contact information redacted, which imply it’s one of the ones explicitly posted for public consumption and representative of 6750 of the 7000 files.

To be crystal clear, Google able to access and continues to host several of the same documents Mr. Big is facing charges over.

The Charges

What are the actual charges? From the Canadian Criminal Code (emphasis mine):

Unauthorized use of computer
342.1 (1) Everyone is guilty of an indictable offence and liable to imprisonment for a term of not more than 10 years, or is guilty of an offence punishable on summary conviction who, fraudulently and without colour of right,
(a) obtains, directly or indirectly, any computer service;
(b) by means of an electro-magnetic, acoustic, mechanical or other device, intercepts or causes to be intercepted, directly or indirectly, any function of a computer system;
(c) uses or causes to be used, directly or indirectly, a computer system with intent to commit an offence under paragraph (a) or (b) or under section 430 in relation to computer data or a
(d) uses, possesses, traffics in or permits another person to have access to a computer password that would enable a person to commit an offence under paragraph (a), (b) or (c).

In order to secure a conviction, the crown would have to prove beyond a reasonable doubt that the access was fraudulent.

Just as this isn’t a new problem, it’s not the first time it’s been before the courts. There are two very high profile cases.

The first, Aaron Swartz, the inventor of RSS downloaded millions of journals from a server at MIT.

“Aaron’s death is not simply a personal tragedy. It is the product of a criminal justice system rife with intimidation and prosecutorial overreach. Decisions made by officials in the Massachusetts U.S. Attorney’s office and at MIT contributed to his death,” his family said.

Sadly, he killed himself while being railroaded by the US justice system.

The second, Andrew Aurenheimer,  was not only charged but convicted of an offence under the US Computer Fraud and Abuse Act. This exploit was almost identical to the FOIPOP issue at hand.

After being sentenced to 3 years in prison, and serving part of it, Aurenheimer’s case took an interesting turn. It was overturned by the US Court of Appeals.

It gets even more interesting, because according to the EFF (emphasis mine)

 Although it did not directly address whether accessing information on a publicly available website violates the CFAA, the court suggested that there may have been no CFAA violation, since no code-based restrictions to access had been circumvented.

The Defense

The question remains, was the access fraudulent?

Remember what I said about the other installations being called “PublicPortal”? And how 6750 of the 7000 records were public anyways, and how this system is literally designed for facilitating “access to information?” Looking at it further, there are no authentication mechanisms, no password protection, no access restrictions. It’s very clear that the software is intended to serve as a public repository of documents.

It’s also very clear that there at least 250 documents improperly stored there by the province. Documents that the province had a responsibility to protect, and failed.

Mr. Big asked for a document, the server returned it, as it’s supposed to. Then asked for them all, and unluckily for him, 250 of the 7000 were “confidential.” He didn’t even try to hide, apparently having been traced by his IP address.

Was that access fraudulent? It’s for the courts to decide, but I would argue no.

Had this system been audited, or looked at by any reasonably competent security professional, this would have been fixed before it became national news and an embarrassment to the province.

An interesting question to consider; was Mr. Big even the only one to discover the flaw? From Global News:

“The employee was involved in doing some research on the site and inadvertently made an entry to a line on the site – made a typing error and identified that they were seeing documents they should not have seen,” Deputy minister Jeff Conrad told a technical briefing.

The government’s official position is that the flaw just happened to be rediscovered last week by a miscellaneous staffer. Apparently, when they raised the issue, the technical team discovered Mr. Big in the logs from a month prior.

They haven’t announced charges against the staffer, so presumably, they don’t consider that manipulation to be “fraudulent.”

Disclosure Theory

I have personally disclosed a vulnerability to the Province of Nova Scotia before, about 2 days before CBC picked up the story of a Russian website broadcasting webcam videos of children in a public school. It was surprisingly difficult to find someone to disclose it to. No one was willing to talk about it, or knew who should handle it. I eventually, via a friend at shared services, got in touch with someone who would take the report. They took it very seriously once the news broke.

To be clear, this is speculation, but it isn’t an unreasonable theory that Mr. Big disclosed the vulnerability to the province. Clumsily maybe, but I honestly believe they tried. I don’t buy the story that the province conveniently happened to discover the breach because someone else noticed the exploit a few weeks later. The system had been in place for over a year and a half, so the timing is suspect at best.

I believe the province failed in their responsibility to protect the data and is now railroading Mr. Big to cover it up.

Since the system is literally designed to serve public documents, the solution to this problem is likely to be costly. It’s easier for the department to blame someone than take responsibility.

In Conclusion

The use of the “Unauthorised Access” statute given the events that appear to have occurred is appalling. The province’s strategy so far has been to cover this up, and when they couldn’t keep it under wraps, bust down some kids door, interrogate him and seize his computers. The charges grossly outweigh the alleged offence, and arguably there was no offence.

I’m disgusted with both HRP and with the crown prosecutors office, for this display of Americanized justice.

If this kid broke the law, so did Google, let alone the giant issue this creates for the information security industry. If discovering a vulnerability can open you up to the same legal liability as manufacturing child pornography, suffice it to say that nothing will ever get disclosed again. Most people aren’t about to risk 10 years in prison to let the province or anyone else know somethings broken. This is generally recognized as a bad thing, weakening security across the board.

Putting confidential documents on a server designed to serve said documents to the public shows a clear lack of judgement, training, and understanding of the software and processes at hand. I think it’s abundantly clear that the blame lies at the feet of the province.

H2FPTF Hackers and green digital computer writing

 

24 responses to “Regarding the Freedom of Information “hack””

  1. […] A script made it easier, but a script wasn’t required. The URLs for FOI documents are incremental. As software engineer Evan D’Entremont points out, anyone could have done what the supposed “hacker” did. […]

  2. Tim Graham says:

    You build a house in the ghetto, but you have to get in to the house, so you take the door off… but now you’re pissed that someone has walked in and taken copies of all your documents?

    Good lord, give me a break.

  3. Andrew Kohlsmith says:

    Personally I would like to see the people responsible for setting up this public portal brought up on charges of gross negligence with respect to protecting the privacy of the citizen’s data which was accessed. They are clearly incompetent and should face the maximum penalty under law for the 250 counts.

  4. […] to privacy lawyer David Fraser and software engineer Evan d’Entremont, you simply had to change the document ID number at the end of a URL and fetch it. So, you’d download document number 1234, then 1235, 1236, […]

  5. […] to privacy lawyer David Fraser and software engineer Evan d’Entremont, you simply had to change the document ID number at the end of a URL and fetch it. So, you’d download document number 1234, then 1235, 1236, […]

  6. […] to privacy lawyer David Fraser and software engineer Evan d’Entremont, you simply had to change the document ID number at the end of a URL and fetch it. So, you’d download document number 1234, then 1235, 1236, […]

  7. […] Alter the document ID number at the end of a URL. Work through all the digits, one by one from let’s say 1234, then 1235, and so on. […]

  8. […] public records containing no personal information. Every request hosted on the server contained very similar URLs, which differed only in a single document ID number at the end of the URL. The teenager took a […]

  9. […] public records containing no personal information. Every request hosted on the server contained very similar URLs, which differed only in a single document ID number at the end of the URL. The teenager took a […]

  10. […] records containing no personal information. Every request hosted on the server contained very similar URLs, which differed only in a single document ID number at the end of the URL. The teenager took a […]

  11. […] public records containing no personal information. Every request hosted on the server contained very similar URLs, which differed only in a single document ID number at the end of the URL. The teenager took a […]

  12. John L says:

    It will be very interesting to see the ITO and arrest warrants and supporting police affidavits. The ISP has to respond to the ITO (Information To Obtain) and the judge or Justice of the Peace who authorized these has to verify the affidavits meet certain standards. Do they meet these standards?

    A few more questions:
    – did the government “IT expert” inform Halifax police of all accesses, including Google and other web scrapers? (Google did capture and make available some of these documents, reportedly. I haven’t confirmed myself).
    – the press have published their own use of the site; were they aware of the URL patterns and had they ever made use of it?
    – is the complete web server access log going to be disclosed to the defence? If not, why not?

    Here in BC it is the Crown prosecutor, not police, who has authority to charge an accused. The Halifax prosecutor may well be in a difficult position as police have apparently charged someone with little regard to current practices in IT. He/she risks putting Nova Scotia under a cloud if they proceed, especially if they are seen to be selective in who they prosecute.

    • evan says:

      In nova scotia it is the police who lay charges and then refer them to the prosecution.

      I’m assuming a lot of this will come out of the woodwork, or they’ll drop the charges for it to come out of the woodwork. I’m going to make another post about what I’ve found since and presented at AtlSecCon. But in short, the Government IT expert was a team including the CIO, CISO, and Deputy Minister of Internal Affairs. And by calling it a hack it drove the response into crazy territory.

  13. […] (For technical details about the portal and what the teen did, check out this post from D’Entremont.) […]

  14. […] (For technical details about the portal and what the teen did, check out this post from D’Entremont.) […]

  15. […] public records containing no personal information. Every request hosted on the server contained very similar URLs, which differed only in a single document ID number at the end of the URL. The teenager took a […]

  16. […] (For technical details about the portal and what the teen did, check out this post from D’Entremont.) […]

  17. […] (For technical details about the portal and what the teen did, check out this post from D’Entremont.) […]

  18. […] (For technical details about the portal and what the teen did, check out this post from D’Entremont.) […]

  19. […] (For technical details about the portal and what the teen did, check out this post from D’Entremont.) […]

  20. […] protection. (Evan d’Entremont, a Halifax-based IT security professional, has provided great coverage of the case and Jacob Boon at The Coast has helpfully pointed out that the security of Nova Scotia’s […]

  21. […] all available documents from the Nova Scotia’s government FOI site — a script that did nothing more than increment digits at the end of the URL to find everything that had been uploaded by the […]

Leave a Reply

Your email address will not be published. Required fields are marked *