TechFails

State of Cybercrime

When technology doesn’t work when it should, is it a tech fail? Or perhaps because humans are creating the technology, fails should be more accurately called a human fail? In this episode, we discuss various types of “fails”, including the latest popular Pokémon Go, why we can’t vote online and the biggest fail of all, a data breach.

• Pokémon Go full access, tech fail or win
• Is it possible to delete an entire company with one line of code?
• Why can’t we vote online?
• Should one person be blamed for a tech fail?
• Technologies that can predict your next security fail
• Parting Gifts

Pokémon Go full access: tech fail or win?

Cindy: This week, I’m calling our show #techfails.

But in preparing for this show and thinking deeply about our fails, I just want to echo what Kilian has been voicing these past couple of episodes, that when our technology fails; like for an instance, if my Skype for business isn’t working, then my first thought is, “Oh, it’s a tech fail. I can’t believe it’s not working.” But we’re the one creating the technology.

So, for me, it feels, at the end of the day, a human fail. Let’s discuss this and debate it for a bit.

To set the context, there was an article in the Harvard Business Review, and eventually turned into a LinkedIn post too. It’s titled “ A New Way for Entrepreneurs to Think About IT.” It said that IT’s primarily known as a necessary evil, IT support or IT as a product. With many different types of technologies at our fingertips, we can really do a blend of both.

For instance, APIs have really changed how firms interact and share information with each other. And we really take this for granted these days, because back then you’d have to get permission from legal to sign contracts before experimenting with partnerships.

Now you can easily partner up with another service within API or use OAuth . It’s really increased our productivity, but it can also have some potential problems if we’re not careful.

For instance, if you downloaded Pokémon Go earlier this week, you might have been given Google full access. That meant that the Pokémon people could read all your emails and send out emails for you.

But since then they fixed it. I think, Kilian, they fixed it pretty quick.

Kilian: Yeah, in about, I think, 24 hours, more or less, they had a patch out that it addressed it already. I think, as opposed to a technology fail, that might be a technology win, for a company really taking these concerns seriously and addressing it as soon as it’s kind of brought up.

Mike: Before we get into that, I just want to know, what’s your guys’ level? How you been doing on Pokémon Go? Have you been getting out there, doing your Pokémon?

Cindy: I’ve been…I actually downloaded it at the office. And I could have thrown something at somebody, but I didn’t. I’m like, “Well, I’m just doing this for work, so better not start running after people and throwing stuff at them.”

Mike: You couldn’t convince the rest of the office that playing Pokémon Go was part of your job?

Cindy: Actually, we had a mobile photography class earlier this week, and Michelle, our HR person, was walking around telling people that Pokémon’s gonna be there. She was doing that for me.

Mike: Nice. How about you, Kilian, have you tried it?

Kilian: No, I haven’t downloaded it. That would require going outside and interacting with things, maybe.

Mike: The first couple ones show up right around you. And I think this is kind of where I was going with this, which is that a lot of this…in terms of tech fails, this is really about managing complexity.

In terms of IT, trying to manage these external services, it’s about managing complexity on an organizational level instead of a personal one. Because when you think about what is involved for this stupid game of Pokémon Go, you’re talking about interacting with geosynchronous orbital satellites for GPS, the internet to get all these apps, these multiple different services. And to pull all that together requires this huge thing. The security issue came about because Google was asking for OAuth access, and that’s just when you use Google to log into it. You log in with your account and it has these things.

And it’s so complex because even though it doesn’t look like it, it actually uses Google Maps data underneath.

A trick you can do, is if you have Google Maps installed on your iPhone, you can enable offline map access. And in order to achieve the app to app communication on your sandbox apps on the iPhone, it needs all these extra permissions, and it’s just insane trying to make that work. It’s so easy when you’re building something to just like, just give me all the permissions, and we’ll slowly back it down until where it’s supposed to be.

Cindy: Do you think this is kind of like, “okay, we’re gonna use external service, and then just not really look at the settings because we’re so focused on making Pokémon Go just a wonderful experience?”

Mike: Well, that’s the consumer side. The level we work at, people try to look at something like Amazon web services, which this article mentions. It is fantastically complex.

It’s something like 60 different individual services that do individual things and also overlap with other ones where like, oh, there’s like six different ways to send an email with AWS. There’s 20 different ways to put a message in a queue to be picked up by something else. Just trying to wrap your head around like, what actually is it doing, is just insane.

And it’s possible to do the stuff. I think it’s just a really hard equation of, “Do we bring this in-house and have a dedicated person for it? Is that more or less of a threat than having this outside?”

Something I see a lot of is…coming more from the app side of things is, people swearing up and down that, “I’m gonna get on a virtual private server somewhere for ten bucks a month, put my own version of Ubuntu on it and keep it up to date.”

And it’s really hard to imagine that that is as secure as having a dedicated security team at AWS or Heroku or one of the other Azure platforms as a service.

It’s that same scenario, sort of, at the organizational level, that either it’s a tremendous amount of effort to maintain and secure all those things yourself, or you’re essentially paying for that in your service contract.

Cindy: I think those are all really good questions to ask, and it requires a huge team.

Is it possible to delete an entire company with one line of code?

Cindy: I kind of want to transition into another fail that’s different than asking good questions and figuring out the architecture.

The next fail is a fail on many different levels. It would be interesting for us to discuss.

Back in April, there was an article published and shared over 65,000 times when a small hosting company with a little over 1,500 users said that he deleted their customer’s hosted data with a single command.

Then later we found out that he was just trying to market his new Linux service for his company. And then people were outraged, “He didn’t do a better job backing up,” they were outraged that he lied to server fault, like a community that really helps one another figure stuff out. It’s security, and backing up, and just technology, it’s complicated.

I was a little skeptical reading the article with the headline that said “One Person Accidentally Deletes His Entire Company With One Line of Bad Code.”

As you’re responsible for hosting data, you should have multiple backups.

One of my favorite comments is, how do you even accidentally type that you accidentally deleted stuff?

What are your thoughts and reactions to this article?

Mike: Kilian, you want to go? I have my own thoughts.

Kilian: Sure.

First off, that’s a terrible job of advertising. I don’t know what he’s advertising for. Like, “Host with us and I might break your stuff.”

I think the point he was probably going for is that it’s easy to make mistakes, so get a dedicated person that knows better.

But I don’t think that really came across.

For the actual command itself, a lot of people are in such a hurry to automate and make things easier that it is easy to make mistakes, especially as Mike mentioned earlier, with these vastly complicated systems with dozens of ways to do the same thing.

The more the complex the system gets, the easier it is to make a mistake. Maybe it could be that disastrous.

But a lot of things really have to go wrong, and kind of poor decisions made throughout the chain. But it’s conceivable that someone could have done that.

Mike: Specifically, to the question that’s asked on server fault, which is like a question and answer side for these issues. There’s a lot of utilities that can either take a single or multiple different directories as arguments.

So you say, “Hey, copy these two things,” or “Copy this one thing.” And so, in this, the person, they put a space so they have like: /pathfolder /. And so, that last slash got interpreted as the root of the volume they were on. And so, hey, we just destroyed everything, and everything includes all your keys and stuff.

Something we talk a lot about in here is layered security, but you need layered backups and recovery as well.

That was really the answer to this, is that they were on a virtual private server.

In addition to just backing up the local data, their database, the files on it, it takes system images of your entire VPS and keeps it somewhere else.

I am incredibly paranoid with backups, especially backups of systems like this. So I always try to even just get it out of the system that…if it’s on…in this case, it was Hetzner, which is a European hosting system, that you get that out onto S3 or you get it out on to Rackspace cloud or something else, just to try to make that a better scenario.

Kilian: That’s a great point, is having multiple different…you can’t have one single point of failure in a system like this.

Otherwise, you could be very vulnerable.

Even for myself when I, for example, backup pictures off of my camera, I have to go to my laptop, I have to go to a network share, and then I have a separate hard drive that I plug in just for that, and then unplug and put it away afterwards. So I have three different places for it. Not that they’re that valuable like a hosting system, but silly things happen sometimes. You know, if I lose power or power surge, I lose two of my systems for some reason, I still have that hard drive that’s sitting in a drawer.

Mike: I have a lot of discussions with people where they have backups and this very elaborate system. They’re like, “All right, I have my local network attach storage here, then I got this ‘nother server, and then I rotate them and do all this stuff.” That’s awesome until their house catches on fire and they lose everything. And that’s the stuff you have to think about. It’s like these things come in in weird ways, especially everything is so interconnected and everything is so dependent upon each other that you can just have these weird cascading levels of failure. And from very crazy sources of stuff. Like, DNS goes like a DNS server gets a DDoS attack. And then that actually ends up taking down like a third of the internet just because everything is so connected.

Why can’t we vote online?

Cindy: Our next fail…I want to know if you guys think that our inability to vote online is a human fail or a tech fail. What do you guys think? Or any opinion, really.

Mike: It’s all in the execution, like all this stuff. That if there was a verifiable, cryptographically secure way of knowing that you could vote, that would be a very positive thing, potentially. It’s a really interesting mix of software and technological concerns, and people, and sociological and political concerns.

What I just said about having almost a voting receipt that says, “Great, you used your key to sign, and you have definitely voted for this person and done this thing.”

One of the reasons that’s never been done, even on most paper stuff, is that that was a huge source of fraud that in like the olden days, when they had voting receipts, you would go and turn them into your councilman and they would be like, “Great, here’s your five bucks for voting for me in this election.”

So that’s just something that’s not done. That’s not a technical issue. It’s certainly possible to do those things, but it leads to all these other unforeseen, I don’t know if you’ve heard of the cobra effect kind of things, these horrible unintended consequences.

Cindy: I think this article on why we still can’t vote online was just very thoughtfully written. It talked about how it can potentially destabilize a country’s government and leadership if they don’t get voting online right. It was really just like, wow, I can’t believe a researcher at The Lawrence Livermore National Lab said, “We do not know how to build an internet voting system that has all the security, and privacy, and transparency and verifiable properties that a national security application like voting has to have.” And they’re worried about malware, they’re worried about ransomware, they’re worried about being able to go in and track, do a complete security audit.

They said something interesting too about how, in the finance system, sure, you have sensitive data, and you can go back and track where the money went more or less, if you have these systems in place. But you might not necessarily be able to do that with voting, and someone can say, “I voted for so and so,” and then change it to somebody else, and they can’t go back and verify that. There are so many elements that you need to consider. It’s not just Pokémon, or you’re not trying to create a wonderful gaming experience, or you’re not trying to back things up. They’re a multitude of things you need to take in to consider.

Kilian: The one big thing, and I think the heart of it, was the need for anonymity in the voting process.

That’s kind of the way it was set up to avoid coercion and some other problems with it, is you need to be anonymous when you cast that vote. By putting it online, the real down side is… Like, if you think about online banking, it’s important to know and verify that you are who you say you are, and have a transaction of that entire process so you can ensure…it’s kind of both parties know that the money transfer from X to Y or so on and so forth. And you have the track of the steps.

But when you try and introduce anonymity into that equation, it completely falls apart. Because if you have that tracking data going back to somebody casting a vote, then they could be a target of coercion or something like that. Or if the opposition party finds out, they could go after them for not voting for whoever.

Cindy: Yeah, they did that with Nelson Mandela.

Kilian: Yep.

And then the other thing too is, as a person casting a vote, if you think about it, you’re kind of trusting the system. It’s completely blackboxed you at that point. So when you click the button and say, “I vote for candidate XYZ,” you have no idea, because, again, you want to be anonymous. You don’t have that verification of the system that says, “Hey, my vote wasn’t changed to candidate ABC in the process.” You kind of have to go along with it.

Even if you look back at some of the physical problems with the George W. Bush election with the ballots not lining up right with the little punches. It was punching for… I forget what the other candidate’s name was.

Cindy: Al Gore?

Kilian: No, no, no. It was like Paton Cannon or somebody. Whoever the third party candidate was. But they were saying, “No, no, I voted for Al Gore…” whoever, but it registered somebody else. They had to go back and manually look at that, and look at the physical paper to see that to validate that. But if you think in a digital system, if you click the button, you have no way to audit that really. Because if the system says, “No, you’ve voted for this guy,” you have no proof, you have no additional evidence to back that up, and that’s the big problem.

Cindy: They actually showed this in “The Good Wife,” the TV show that is no longer around, or they just ended. The voters would go in and they would vote for someone, but then it would also give the other person five additional more votes. I think another thing to…they didn’t mention it, but I think politicians or just that kind of industry are kind of a tad bit slower in the technology side.

Because Barack Obama’s campaign really set the tone for using technology and using social media to kind of engage the voters. It’s kind of like he really changed how now politicians are marketing and connecting with people. I don’t know, do you feel like they’re kind of behind? Or maybe that’s just me?

Kilian: My personal opinion is, we have laws that don’t make sense with where technology’s at, because they are slow. We’re still running on laws, and been prosecuting cases with laws that were made in the ’80s and early ’90s, and even older in some cases, where technology was vastly different than what we have today. This might be off topic, but there was just, I think, a ruling that the Computer Fraud and Abuse Act could theoretically mean that if you share your Netflix password, it’s a federal crime. Now, that’s open to interpretation, but that was a story I had seen the other day. We have all this technology and it’s evolving much, much faster than the people making the regulations can kind of keep up with it.

Mike: I just want to see a Poke stop at every voting registration.

Cindy: Mike has Pokémon on his mind.

Kilian: It’s great, it’s good fun.

Cindy: Now I have Pokémon…I actually visualized us playing Pokémon at a voting station. That would be interesting. It’s too hot and humid in New York to do that.

Kilian: Vote to vote or play Pokémon.

Cindy: I almost want to say Poke because it’s so hot.

Kilian: Well, to the candidates out there, the first one to get on top of this making a Poke stop at the voting booths in November might seize the election with the youth vote.

Mike: A Pokémon at every pot.

Should One Person Be Blamed For A Tech Fail?

Cindy: Let’s also kind of think about potential fails, though. We’ve seen Target, Sony, the data breaches. And so, when fails happen that costs them their jobs, do you think one person should be blamed for all of it or can we also kind of say, “We don’t have the technology right yet”?

Mike: It’s interesting. What we’re talking about is, there have been a lot of very large data breaches. And what seems to happen is, it happens and then depending upon how much press it gets, the CEO has to resign or doesn’t. Or in the case of the OPM, the director. The parallel that I like to think of is Sarbanes Oxley, which has had a lot of other consequences. But the big one was that the chief executive has to sign off on the financials of the company. Before, it was always there were a lot of scandals where it was like, “I’m just running the company. My CFO and the accounting group, they were doing their own thing with the funds. And I wasn’t aware that this…”

Then we said this like 10,000 pounds of coconuts we had on the dock, they were rotten were actually good. We counted those in the asset, all of those kind of shenanigans. And just that thought that, okay, the finances and the statements that are put out, that is an executive level sign off, that there’s a responsibility at that level to ensure that those are correct. What we’re seeing is sort of that happening on the IT security side. That maintaining integrity of your customer’s data, of the people you’re responsible for, that is something that the executives need to say is a priority, and to ensure that in any way they can. That if they aren’t doing that, that’s their job, that they failed at their job.

Now, looking through these kind of stories, you typically find that the person in charge is not a network security person, because there’s not a lot of people that get their CISSP and then say, “I’m qualified to be CEO.” That’s just not how the normal job progression works. But they need to have people in place, and they need to make sure that the right things are happening, despite not having the personal expertise to implement those but that they make it a priority and they give budget, and they’re able to balance it against the other needs of the company.

Technologies that can predict your next security fail

Cindy: In order to come back from a security or technology fail…there was an article about “There’s new technology that can predict your next security fail.” They are essentially talking about predictive analytics. I really like a quote that they wrote that, “It’s only as good as the forethought you put into it, and the questions that you ask of it.”

If you don’t think about it, if you don’t have a whole team to work on this huge security and technology problem…because there’s only so much you can…in terms of big data, machine learning, predictive analytics, there’s a lot of stuff, a lot of elements that you’re unable to kind of account for.

So if you don’t consider all the different elements in security, you can’t build that into the technology that we build. What are some other things you think that can help companies prevent or come back from a tech fail or a security fail or a human fail?

Kilian: The only thing I could get in my mind there was asking the right questions. For me is from Hitchhiker’s Guide to the Galaxy. If you ask it, what’s the meaning of life, the universe and everything, it’s gonna give an answer. But what’s the question you’re really trying to get out of it? That’s all I can think of in my head. I think that’s one thing people get stuck in a lot of times, is asking the wrong questions that they need from their data. I’m sorry, Mike, I cut you off there. You were gonna say something.

Mike: I’m in agreement with you, Kilian, because I think too often the question posed is, “Are we secure?” There’s no crisp answer to that. It’s never gonna be yes, we’re 100% good, because the only way to do that is not to have any data, and not to have any interactions with customers. If that’s the case, then you don’t have a business. So you have to have something. You still have to have people interacting, and the moment you have two people interacting, you’re vulnerable at some level. They can be tricked, they could do anything. And then you have networks, and the networks are talking.

So it’s much more about, what is the level of risk that you find acceptable? What steps can you take towards mitigating known dangers? How much effort and time and money can you put behind those efforts? There’s no quick fix. Something we talk about a lot on this is that data is, in a lot of ways, like a toxic asset. It’s something that you need to think about like, “Oh, we have all this extra data. Well, let’s try and get rid of some of it. Just so we don’t have it around to cause us a problem, just so we don’t have it around to be leaked in some way.” There’s lots of different ways to do that and lots of benefits of doing so.

Parting Gift

Cindy: Now in the parting gift segment of our show, where we share things we’re working on, or something we found online that we think our viewers and listeners would appreciate. I just read that Chrysler, the car brand, is offering a bug bounty between $150 to $1,500 for finding bugs. But you can’t make it public. And also, I just updated top InfoSec people to follow. I included a whole bunch of other women that were missed. So check that out at blog.varonis.com.

Mike: Who’s the one person you think we should follow that we weren’t before?

Cindy: I definitely think we should be all following Runa Sandvik. She’s the new InfoSec security person. She writes about the Info security at the New York Times. She also worked on Tor, and she did this really cool rifle hack. And she wrote about that. Or someone wrote about her hack on Wired. Any parting gifts, Mike?

Mike: I was gonna recommend Qualys’ SSL lab server test. If you’re unaware of what it is, you can put it in your website and it will run through all the different ways in which you’ve screwed up setting it up properly to be secure. It gives you a nice letter grade. So, a couple interesting things about this. One: It’s really hard to make one of these yourself, because to do so, you have to maintain a system that has all of the old, bad libraries on it for connecting on SSL1 and 2 and 3 that are deprecated. Just so you can make the connections and say, like, “Yes, this remote system also connects with this.” So it’s not something you want to do, and it’s not something you can do trivially. So it’s great that this is an online service.

And then two: I think it’s really interesting how…they essentially just made up these letter grades for what they consider as an A, A+, B. But in doing so, they were able to really improve the security of everyone. Because it’s one thing to say, “Okay, out of 200 possible things we comply with, 197 of them.” It’s a different thing to know, “Okay, we got a failing grade because one of those three things we didn’t do was actually really, really bad and exploitable.” And to be able to compare that across sites, I think, just has a lot of incentives to make everyone improve their site. Like, “Oh, gosh, this other site is a better grade than us. We should definitely improve things.” So for those reasons, I think it’s a really great part of the security ecosystem and a great tool for all of that.

Cindy: Kilian, do you have a parting gift?

Kilian: I was reading an article the other day, it was pretty interesting how we all come to rely on our phones and our digital assistance, like Siri or Google Now, to make our lives easier to interact with a device. Some researchers started thinking that, “Hey, this is a good avenue for exploitation.” They started kind of distorting voice commands so they can embed it in other things, to get your phone to do stuff on your behalf. So, it’s just an interesting thing to keep aware of and how you’re using your digital assistance, because other people could start to exploit it by issuing voice commands to it to maybe direct you to a malicious site or something. It’s one more thing to kind of keep in the back of your mind.

---

Subscribe Now

Join us Thursdays at 1:30ET for the Live show on Youtube, or use one of the links below to add us to your favorite podcasting app.

• iTunes
• Android
• RSS

The post TechFails – IOSS 15 appeared first on Varonis Blog.