Results 1 to 21 of 21
  1. #1
    Thailand Expat tomcat's Avatar
    Join Date
    Nov 2005
    Last Online
    @
    Posts
    12,663

    Unusual Ransomware Recovery

    How to Survive a Ransomware Attack Without Paying the Ransom

    Norsk Hydro used faxes, Post-its, and old PCs to beat cybercriminals.


    Michael Hammer, Norsk Hydro’s plant manager in Cressona, Pa.
    PHOTOGRAPHER: WILLIAM MEBANE FOR BLOOMBERG BUSINESSWEEK
    By William Turton
    July 23, 2020, 11:01 AM GMT+7

    At around midnight Oslo time on March 19, 2019, computers owned by Norsk Hydro ASA, a large aluminum manufacturer, started encrypting files and going offline en masse. It took two hours before a worker at its operations center in Hungary realized what was happening. He followed a scripted security procedure and took the company’s entire network offline—including its website, email system, payroll, and everything else. By then, a lot of damage was already done. Five hundred of Hydro’s servers and 2,700 of its PCs had been rendered useless, and a ransom note was flashing on employees’ computer screens.

    “Greetings!” the note began. “There was a significant flaw in the security system of your company. You should be thankful the flaw was exploited by serious people and not some rookies. They would have damaged all your data by mistake or for fun.” The message instructed recipients to write to an email address to discuss an unspecified payment, which would have to be made in Bitcoin; in exchange, the hackers would provide an encryption key to reverse the damage.

    Like most other large multinationals, Hydro had been at least aware of the possibility of attack. It had a cyber insurance policy, and it had tested its networks with “white hat” hackers—security consultants who attempt to break into a system to check its defenses. “I wouldn’t say we could keep the NSA out,” says Chief Information Officer Jo De Vliegher. “But we were a company with all the normal security in place.”


    De Vliegher COURTESY: NORSK HYDRO
    It wasn’t enough. Some 35,000 employees were locked out of the company’s network, and Hydro had to shut down several manufacturing plants in Europe and the U.S. The ones still operating had to figure out how to do so without any computers. In the end, the attack would cost the company more than $60 million—way more than the $3.6 million the insurance policy has paid out so far, according to an earnings report. It was, according to the prosecutor investigating the breach, the worst cyberattack in Norway’s history.

    Despite all this, Hydro never considered paying the ransom, because the anonymous hackers could have just taken their Bitcoin and disappeared. Even if they’d provided the key—and even if the key worked—it would have sent a message that Hydro was an easy mark, leading to future attacks and more extortion.

    Instead, De Vliegher oversaw a fitful recovery from the attack, improvising with ancient PCs, fax machines, Post-it notes, and all manner of other analog technology. The response illustrates the painful reality that security consultants and law enforcement officials often bring up: Even when you do everything you can to protect yourself from a cyberattack, a determined adversary will almost always be able to wreak havoc. In other words, it’s less a question of how to stop hackers from breaking in than how to best survive the inevitable damage.

    On the night of the attack, De Vliegher had just landed in Belém, Brazil, where Hydro has a large presence. As soon as he heard computers had been encrypted, he took the first flight home. By the time he made it back to Hydro’s corporate headquarters in Oslo, a team of five specialists from Microsoft Corp. was there, working to diagnose the problem and figure out how to restore the company’s data. Employees had taped handwritten notes to the doors warning others not to turn on any phones connected to the company network.

    Hydro needed to alert customers, suppliers, employees, and investors, but the company’s website was down. So at 9:42 a.m. the day after the hack, an employee on the communications team used his personal cellphone to make a post on the company’s Facebook page: “Hydro is currently under cyber attack. Updates regarding the situation will be posted on Facebook.”

    Next, Hydro had to make sure employees got paid. Banks were refusing to communicate digitally with the company, fearing that whatever had infected its network would spread to them next. Payday in Brazil was two days away, and 5,000 employees there were expecting a check. De Vliegher came up with a solution: He copied the previous month’s paychecks from an external payroll system, removing the employees who’d been fired or quit in the meantime. “It was about 90% accurate,” he says.

    Shipping and receiving in Cressona. PHOTOGRAPHER: WILLIAM MEBANE FOR BLOOMBERG BUSINESSWEEK

    Of all the many operations Hydro has around the world, from the bauxite mines in Brazil to the hydroelectric power plants in Norway (hence the name), the damage was worst in Cressona, Pa., where the company operates its largest aluminum plant. The Cressona facility was built by the U.S. government during World War II to make aluminum for weapons; it has a sawtooth roof that was designed to confuse enemy bombers into thinking they were looking at ripples on a lake. The plant is run by Michael Hammer, who started there 25 years ago in accounting and stayed on as it was passed among different owners. (Hydro acquired Cressona in 2017.)

    It was dinnertime in Pennsylvania on March 18 when Hammer got a call from Hydro’s vice president for risk management. “Get your folks to the plant,” he remembers the VP saying. “Print out as much stuff as you possibly can before they start pulling the plug on the servers.” Hammer had experienced brief outages before. Maybe someone down the road ran their car into a power line, he thought, figuring the plant would come back online in a few hours.

    He knew it was bad as soon as he arrived and saw workers frantically unplugging computers. Then he read the ransom note. “I didn’t even know what the hell Bitcoin was,” he says.

    The Cressona plant. PHOTOGRAPHER: WILLIAM MEBANE FOR BLOOMBERG BUSINESSWEEK

    Under normal circumstances, his plant employs 1,180 people, runs 24/7, and produces more than 2.6 million pounds of finished aluminum a year. Walking through it today, you can feel the heat from the furnaces where recycled metal is melted down and reformed into large cylinders. These are heated and pushed through 60-pound circular dies, transforming them into components for such products as window frames and flooring. Imagine pushing Play-Doh through a cookie cutter. Customers include Tesla Inc. and Ford Motor Co.

    This kind of manufacturing predates computers, but computers have made it much more complex. Hydro has more than 50,000 dies, and it uses software to keep track of what’s being made and to tell employees which die to pick off the shelf. Without access to customer orders, technicians had no idea what to make. Hydro employees began calling customers, asking them to text or send orders to personal email accounts. With the corporate email system down, plant staff traded phone numbers and communicated by group text.

    A temporary war room in Cressona. PHOTOGRAPHER: WILLIAM MEBANE FOR BLOOMBERG BUSINESSWEEK

    As the orders started to trickle in, the only way for people on the plant floor to know what to do was by reading off a paper copy of each order. Luckily the plant had a bunch of old computers in storage, which Hammer set up in a war room to print the forms. “We went over to Staples, and we pretty much cleaned them out of printers and paper and cartridges,” he says. Salespeople, whose computers were also hacked, had nothing to do, so Hammer had them strap on safety gear and run paper orders to workers on the plant floor.

    Printed work orders kept the factory running while the network was down. PHOTOGRAPHER: WILLIAM MEBANE FOR BLOOMBERG BUSINESSWEEK

    For the first week, Hammer lived at the plant, occasionally taking naps on a couch in his office. Losing access to Hydro’s network also meant he wasn’t able to pay his monthly bills to suppliers, and they were calling to ask where their money was. So he pulled an old fax machine out of a closet and asked suppliers to fax payment details, which he then forwarded to Hydro’s bank. The suppliers who still had fax machines lying around got paid first.

    Hammer is still searching for answers as to who could have attacked his plant and gotten away with it. “It was a lot of manual stuff, a lot of long hours, a lot of long days,” he says. “And that pain was injected by an evil person. It was a terrorist basically. And what made it worse is it was nameless, faceless. You don’t know where it came from, how it got there.”

    Hammer PHOTOGRAPHER: WILLIAM MEBANE FOR BLOOMBERG BUSINESSWEEK

    Nobody has figured out who attacked Hydro, but signs point toward an organized cybercrime group operating with impunity somewhere in Eastern Europe. The group made headlines last year for hacking point-of-sale systems to steal credit card numbers. Known to security researchers as FIN6, it’s often extracted Bitcoin ransoms in the hundreds of thousands of dollars. “Fin” is short for “financially motivated,” to differentiate the gang from military hacking units affiliated with countries that have active cyberweapons programs, including China, North Korea, Russia, and the U.S.

    FIN6’s signature weapon is a virus called LockerGoga, named after one of the files buried in its malware. There are dozens of variants of the software, and Hydro thinks the attackers deployed more than one within its network, making it harder to expunge from the company’s systems.

    Ransomware hackers generally penetrate computers more or less at random, then use a self-propagating software program—a worm—to work their way deeper into the corporate network. But in Hydro’s case, the attackers gained access by hijacking a legitimate email from an Italian customer. The customer had attached a file, which the hackers modified. When the file was opened, on Dec. 5, it executed malicious code, allowing the invaders access to the entire network. They waited until March to launch their attack. The company doesn’t know if the hackers first compromised the customer or if the message was intercepted and changed in transit.

    Hydro wasn’t the first industrial company to be hit by the LockerGoga virus. A French engineering company, Altran Technologies SA, was struck in January 2019. Later that year, U.S. chemical companies Hexion Inc. and Momentive Performance Materials Inc. received copies. Large industrial companies aren’t conventional ransomware targets, leading some computer security researchers to wonder if the attacks were about sabotage rather than greed.

    In addition to encrypting Hydro’s computers, the virus changed the password of every administrator account, logged those accounts out, then restarted each computer, making it harder for employees to even see the ransom note—which didn’t include a specific demand for money, or even the address of a Bitcoin wallet. There was just an email address. Of course, these idiosyncrasies could have been dreamed up by FIN6 to make Norsk executives feel more vulnerable, says Charles Carmakal, senior vice president for cyber[at]security firm Mandiant. Norsk says there’s no evidence the hackers wanted anything other than money.

    Investigators at Kripos, Norway’s equivalent of the FBI, and Europol, the EU’s law enforcement agency, are still sifting through terabytes of data from the hack. They’re not especially optimistic about making an arrest. Cyber[at]crime groups use encrypted apps and take payment in cryptocurrency, making traditional policing tools, such as wiretaps and search warrants, useless. On top of that, the cross-border nature of crime creates mountains of paperwork to retrieve evidence that may be stored on servers in another country. “The criminals can communicate freely without law enforcement being able to read what they are saying,” says Knut Van Jostein, the prosecutor leading the investigation.

    Back at Hydro’s headquarters, the emergency response team spent weeks locked inside a conference room as they rebuilt the entire network from scratch. They were para[at]noid about any further intrusions, so even the cleaning staff was barred from entering. De Vliegher says the room got very messy. “This is the most secure room we have, so we don’t want anyone to leave whatever spy pens and microphones and stuff behind,” he says in an interview in Oslo.

    Signs reminded employees not to use devices connected to the network. PHOTOGRAPHER: WILLIAM MEBANE FOR BLOOMBERG BUSINESSWEEK

    Recovery meant creating a safe zone of computers that definitely didn’t have the virus and slowly moving other machines that had been verified as clean over to the new network. Progress was slow. Three weeks after the attack, Hydro had a total of four functioning PCs in all of the U.S.

    Employees in France set up a make-shift assembly line to build new, noninfected PCs, and created a sort of bucket brigade to transport PCs across Europe. Workers drove to a gas station in the middle of the country to swap infected computers for clean ones. At a plant in Magnor, east of Oslo, pensioners who lived nearby came out of retirement to help with printing and sorting orders.

    Hydro executives are grateful the loss was just $60 million. In the darkest days following the hack, some feared they’d fall so far behind on orders it would sink the entire company. “We came out of it stronger because of all the 35,000 people that worked overtime, weekends, changed jobs. Nobody complained,” De Vliegher says. “But in a company where that willingness is not there, it’s lethal.”

    Things were mostly back to normal when a Bloomberg Businessweek reporter visited last September, but the company still hadn’t fully recovered. In Magnor, employees had lost access to the software that runs its production line. Luckily, a similar plant in Denmark was spared, and an employee there sent a copy of the program on a flash drive. The staff electrician in Magnor, who moonlights as an IT support guy, figured out how to install thenew copy. The software works well enough, though it’s all in Danish.
    Last edited by tomcat; 25-07-2020 at 08:28 AM.
    Majestically enthroned amid the vulgar herd

  2. #2
    En route
    Cujo's Avatar
    Join Date
    Jan 2006
    Last Online
    Today @ 04:07 PM
    Location
    Reality.
    Posts
    30,575
    Well that was boring.

  3. #3
    Thailand Expat TheRealKW's Avatar
    Join Date
    Aug 2019
    Last Online
    @
    Posts
    3,776
    I found it interesting, the use of analogue machines and private emails helped the company survive. I wonder if they will set up a redundancy system for next time.

  4. #4
    disturbance in the Turnip baldrick's Avatar
    Join Date
    Apr 2006
    Last Online
    Today @ 05:56 PM
    Location
    Heidleberg
    Posts
    22,544
    Quote Originally Posted by TheRealKW View Post
    the use of analogue machines
    I missed that

    what analogue machines did they utilise ?

  5. #5
    Thailand Expat tomcat's Avatar
    Join Date
    Nov 2005
    Last Online
    @
    Posts
    12,663
    Quote Originally Posted by baldrick View Post
    what analogue machines did they utilise ?
    ...post-it notes...

  6. #6
    Super Expat
    Plan B's Avatar
    Join Date
    Jul 2006
    Last Online
    Today @ 04:59 PM
    Location
    Inside your head
    Posts
    5,097
    ^^

    They counted on their fingers, and for bigger orders, even their toes.

  7. #7
    Custom user Neverna's Avatar
    Join Date
    Mar 2012
    Last Online
    @
    Location
    Behind a rhododendron bush
    Posts
    19,098
    March 2019
    Plant manager, Michael Hammer: “I didn’t even know what the hell Bitcoin was”.

  8. #8
    Thailand Expat harrybarracuda's Avatar
    Join Date
    Sep 2009
    Last Online
    @
    Posts
    69,051
    Three weeks after the attack, Hydro had a total of four functioning PCs in all of the U.S.
    That is a pitiful response.

  9. #9
    Super Expat
    Plan B's Avatar
    Join Date
    Jul 2006
    Last Online
    Today @ 04:59 PM
    Location
    Inside your head
    Posts
    5,097
    A firm this size should have offsite data backups and a disaster recovery process.

    Looks more like someone dropped the ball in management than a successful recovery from a cyber-attack.

  10. #10
    Thailand Expat
    OhOh's Avatar
    Join Date
    Jul 2010
    Last Online
    Today @ 05:55 PM
    Location
    Where troubles melt like lemon drops
    Posts
    20,470
    Quote Originally Posted by Plan B View Post
    someone dropped the ball in management
    Or the insider knew the specifics of the system, what the white hats has tested and documented ....

  11. #11
    Thailand Expat harrybarracuda's Avatar
    Join Date
    Sep 2009
    Last Online
    @
    Posts
    69,051
    Quote Originally Posted by Plan B View Post
    A firm this size should have offsite data backups and a disaster recovery process.

    Looks more like someone dropped the ball in management than a successful recovery from a cyber-attack.
    Absolutely.

    Obviously a complete shit show from start to finish.

    FFS you could send Buttplug to Best Buy and have four functioning computers by the end of the day.

    Well OK, maybe I'm exaggerating.

  12. #12
    disturbance in the Turnip baldrick's Avatar
    Join Date
    Apr 2006
    Last Online
    Today @ 05:56 PM
    Location
    Heidleberg
    Posts
    22,544
    ^ would you like regedit with that ?

  13. #13
    Thailand Expat lom's Avatar
    Join Date
    Jan 2006
    Last Online
    Today @ 05:56 PM
    Location
    Trapped in an old mans body
    Posts
    9,591
    Quote Originally Posted by harrybarracuda View Post
    FFS you could send Buttplug to Best Buy and have four functioning computers by the end of the day.
    with Win XP SP1 and 2GB RAM

  14. #14
    Thailand Expat
    thailazer's Avatar
    Join Date
    Jul 2010
    Last Online
    Today @ 09:57 AM
    Posts
    2,134
    Quote Originally Posted by Plan B View Post
    A firm this size should have offsite data backups and a disaster recovery process.

    Looks more like someone dropped the ball in management than a successful recovery from a cyber-attack.
    Even offsite back ups are at risk as some of the hacks involve delayed execution. The hackers can wait several months until all the back ups have the malware in place.

  15. #15
    Thailand Expat harrybarracuda's Avatar
    Join Date
    Sep 2009
    Last Online
    @
    Posts
    69,051
    Quote Originally Posted by thailazer View Post
    Even offsite back ups are at risk as some of the hacks involve delayed execution. The hackers can wait several months until all the back ups have the malware in place.
    "Delayed execution" of what?

  16. #16
    Thailand Expat
    thailazer's Avatar
    Join Date
    Jul 2010
    Last Online
    Today @ 09:57 AM
    Posts
    2,134
    Quote Originally Posted by harrybarracuda View Post
    "Delayed execution" of what?
    It can be a variety of things. Lots of people get viruses and key-loggers that show up immediately, but some of the ransom-ware malware programs will just infect your directories and back-ups until a certain amount of time passes before they alert the user they have been had. An IT security conference I went to last year focused specifically on how to avert those types. Not an easy thing to do and prevention is better than recovery. Imagine having a years worth of unconnected off-site back ups and they are all infected.
    Press On Regardless

  17. #17
    disturbance in the Turnip baldrick's Avatar
    Join Date
    Apr 2006
    Last Online
    Today @ 05:56 PM
    Location
    Heidleberg
    Posts
    22,544
    considering they had been inside the network for 4 months they could have been encrypting the backups and reporting that all was good

    maybe users' email should be running in a remote sandbox which does not allow them to save files locally without approval

  18. #18
    Thailand Expat jabir's Avatar
    Join Date
    Jul 2016
    Last Online
    @
    Posts
    11,325
    Anything could true or bs but I read some years back hackers accessed Hansard backups, removing random numbers of words, lines, paragraphs, pages and dates.

  19. #19
    Thailand Expat harrybarracuda's Avatar
    Join Date
    Sep 2009
    Last Online
    @
    Posts
    69,051
    Quote Originally Posted by thailazer View Post
    It can be a variety of things. Lots of people get viruses and key-loggers that show up immediately, but some of the ransom-ware malware programs will just infect your directories and back-ups until a certain amount of time passes before they alert the user they have been had. An IT security conference I went to last year focused specifically on how to avert those types. Not an easy thing to do and prevention is better than recovery. Imagine having a years worth of unconnected off-site back ups and they are all infected.
    If someone can infect you with ransomware and you don't notice, you don't have proper controls.

    If your backups are corrupted or encrypted, and you don't notice, you're not doing proper restore drills.

    And if you are backing up viruses and keyloggers, your CISO needs the boot.

  20. #20
    disturbance in the Turnip baldrick's Avatar
    Join Date
    Apr 2006
    Last Online
    Today @ 05:56 PM
    Location
    Heidleberg
    Posts
    22,544
    Quote Originally Posted by baldrick View Post
    4 months
    December 5th until march

    they were more open than butterfluffers rectum

  21. #21
    Thailand Expat harrybarracuda's Avatar
    Join Date
    Sep 2009
    Last Online
    @
    Posts
    69,051
    I wonder how long they had Domain Admin without anyone noticing?

    Schoolboy error these days.

    Unusual Ransomware Recovery-untitled-jpg

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •