Skip to content
  • Home
  • Videos
  • Ransomware Took Down Active Directory • SHIFT On-Demand

ON-DEMAND SESSION

Ransomware Took Down Active Directory. Here’s How JTEKT Recovered

JTEKT shares a firsthand account of restoring Active Directory after ransomware, including the clean-recovery decisions, recovery priorities, and lessons that reshaped its readiness.

Key Takeaways

Immutable Is Not Automatically Clean

Immutable backups can preserve compromised data, making trusted recovery-point selection essential.

 

Identity Recovery Comes First

Active Directory recovery restores trusted access before broader production systems return.

 

Persistence Can Survive Restoration

JTEKT restarted recovery after discovering command-and-control persistence in the restored environment.

 

Priorities Need One Plan

A defined recovery hierarchy prevents competing requests from disrupting the response.

 

Test the Full Recovery

Runbooks and test restores help teams practice complex forest and controller recovery.

 

Data Discipline Affects Speed

Large, unmanaged backup sets can slow restoration when every hour matters.

About This Session

JTEKT’s ransomware recovery exposed the challenge of restoring trusted identity, data, and operations under intense pressure. The company had immutable backups, incident response and communication plans, but the real event showed that recovery depends on practiced roles, trusted recovery points, disciplined priorities, and confidence in the plan.

Finding a Clean Recovery Point

The first restoration contained command-and-control persistence, so the team shut systems down again and examined backups more carefully. The clean recovery point was ten days old, affecting new hires, permissions, data, and other recent changes.

Immutable does not mean clean; teams need evidence that a recovery point predates compromise and removes persistence. JTEKT restored a primary domain controller in isolation, cleaned the environment, and forced password changes across the organization.

Recovering in Business Order

JTEKT restored corporate communications first, production next, then warehousing and sales. The sequence reflected automotive manufacturing, just-in-time supply chains, and contractual obligations.

The team followed one recovery hierarchy instead of allowing urgent requests to continually redefine priorities during the incident. At one point, the two-person recovery team faced 27 number-one priorities. Test restores and clear runbooks build the muscle memory needed to execute under pressure.

Data Management Matters

Restoring a 12-terabyte backup over the network slowed recovery. Retention policies and data stewardship can reduce unnecessary volume and keep teams focused on critical information.

Related Resources

ASSESSMENT

Identity Resilience Assessment

Evaluate your AD readiness to withstand and recover from incidents.

Take the assessment about Identity Resilience Assessment
SELF-GUIDED TOUR

Identity Resilience Self-Guided Tour

Take a self-guided tour of a real Active Directory attack scenario.

Take the tour about Identity Resilience Self-Guided Tour
SOLUTION BRIEF

Protect and Recover AD

Prepare, respond, and recover from an attack on Active Directory.

Read the brief about Protect and Recover AD

Frequently Asked Questions

Why can immutable backups still contain ransomware?

Immutability prevents backups from being changed or deleted, but it does not verify that the protected data is clean. If compromise began before a backup was created, the immutable copy may preserve malicious files, configurations, or persistence.

Why is Active Directory recovery a priority?

Active Directory controls authentication, permissions, and access to business systems. Recovering a trusted identity foundation helps organizations restore communications and applications without reconnecting compromised accounts or domain controllers.

What happened during JTEKT’s first restoration attempt?

The team discovered command-and-control persistence remained in the restored environment. It took systems back down, examined backups with greater precision, and selected an earlier recovery point that was considered clean.

How should organizations prioritize ransomware recovery?

JTEKT followed a recovery hierarchy: restore corporate communications, then production, followed by warehousing and sales. A documented sequence helps teams resist competing urgent requests and focus limited resources on business-critical services.

How can teams prepare for Active Directory recovery?

Create recovery runbooks, define decision owners, test domain-controller and forest restores, validate recovery points in isolation, and practice communication procedures. Regular exercises help responders follow the plan when time and pressure increase.

Transcript

View full transcript

Welcome to this SHIFT customer podcast. I’m your host, Chris Mierzwa as well with Commvault, and I’m incredibly excited to be joined by Billy Gerald from JTEKT. And we are going to talk about, of course, ransomware and an incident that happened.

And, Billy and his company have been very, very kind to share their story backdrop, what happened and more importantly, lessons learned, recovery and everything that comes along with that. So and he doesn’t know this, but I’m a huge fan of Japan and they are a Japanese owned company.

Because a lot of folks may not know the history. Sure, absolutely. And, Chris, thank you and Commvault for inviting JTEKT to participate in the events here at Shift this year. It’s been, exciting

I’m Billy Gerald, I’m the senior manager of operations for JTEKT North America. We have operations in Canada, the US, Mexico, Panama, Brazil, Argentina. So, it’s a fairly large, organization that we cover here in North America.

We’re an automotive manufacturer. We manufacture steering and driveline equipment for most of the major automotive manufacturers in North America, as well as, we do bearing manufacturing and distribution. Well, needle classic needle bearing, the tapered roller bearings that go and transmissions, thrust bearings. We have that side of the business as well.

Yeah. I was really excited to research for this, I went and watched a few other videos and I don’t know who did it, Martin Scorsese or whatnot, but they are well done, well narrated, and some incredible zoom ups on drive shafts. I mean, so if you’re in the motors and you’ve got whoever did all that did a great job.

But it’s, it’s really difficult to make manufacturing exciting at times. And we’re gonna make that drive shaft. But it’s a critical component. And, y’all are at the top of your game and doing it. We are. We are a world leader in manufacturing of those components. You know, I, I wish we made more sexy type things, but we make we make the things that are necessary.

And the engine or the wheels and, wheels turning. Yes, absolutely. So, we’re going to talk about when the wheels don’t go in motion. Absolutely. And maybe you can take us back, turn the Wayback Machine to when you knew there was an identity issue. And maybe then part B of that when you when you did that discovery, when that did happen, what were the first steps or, you know, walk us through that kind of take us into the room with you guys?

Sure. Absolutely. So, to give a little bit more, perspective around it, you know,

JTEKT and, and our team, we have a fairly good resilience platform. You know, we had the immutable backups, we had the incident response plan, we had the communication plan. We had, you know, all of the pieces in place, but maybe not as well oiled as they should have been.

Okay. We just had like rehearsed the muscle memory, the muscle memory type, rehearsed training, engagement. As I had mentioned, you know, previously in some conversations, when you look at the size of the organization, and we look at the recovery team, it was just me and another guy, you know, two people doing the recovery.

So, it was it was somewhat of a challenge, but we did make it through, in the beginning. So, we experienced our cyber event October 10th of 2024. Yeah. Started around 7:38 p.m. There are a couple of, cybersecurity firms that do analysis on this type of stuff. So, Sophos is one of them that does the active adversary report that they publish, mid trend.

Mandiant is another one. They publish statistics as well. That’s owned by Google Cloud. 88% of the cyber attacks happen after hours or so. Of course, I had to get home for dinner. You know, your alertness is down, your awareness is down. It’s turned over to third party, you know, monitoring.

If you have that, capability, a SoC. We were using a SoC. Absolutely. We were using a SoC. You know, 88% of those, cyber events happened after business hours, from the time of threat actor. Another statistic. And by the time of threat actor enters your system, it’s an average, you know, a median of 3.4 hours before they obtained access to your Active Directory.

You know, that’s a very short period of time. Threat actors are normally in your environment for a median of nine days. You know, some are lower, some are more. So, a lot of what’s been discussed in, the shift event is what do you trust, right? Trust is a big thing. So, we had to understand when they infiltrated how long they had been in our system.

What backups do we trust? We’ve got immutable backups. So we’re, you know, we’re making immutable even the bad stuff as well, by the way. That is, if fantastic point, right? Not to pause the story. You can do all of that. You can do all of that understanding that you had probably made immutable and indelible, both good and bad things is something we we talk about in workshops.

Tabletop six seven. So that’s great that you knew that. And you’re like, look, we got to keep our eyes wide open.

So where do you dial it in? Right? I mean, how do you go back and look and see when your system was infiltrated? Because the last thing you want to do is start a recovery process and then have to start all over again.

And I know that because that’s exactly what we did. Right? So, we looked at, the history of the infiltration, and we looked at some of the anomalies that were in our system, and we chose a date. We said, you know, this looks good. Okay. And so, we started restoration activities, moving along and I’ll get to some other details of that path as well.

So, we started the restoration activities. Come to find out that we had command and control still in our system. We still had persistence in our system. So, we had to take everything back down and start combing through those backups a little with a little more granularity. And all of this takes time. The whole time actually looking and seeing, okay, is this clean?

Is it not? And methodically. And what, you know, what are the anomalies that we’re seeing? Are they expected anomalies? Are they just one offs? Is it actually someone that’s taking control of our or, you know, domain controllers? What can we trust? And I think that’s a big key point, that’s been brought about here and SHIFT so we started the recovery.

We picked a day. It was ten days from the attack, ten days back. Right. So, the challenge with that is you lose ten days in new hires. When you’re restoring an Active Directory, you lose ten days of additional permissions that may have been given to current users. You lose ten days of data. You lose ten days. Can your organization manage ten days’ worth of loss and you actually didn’t? Do you have a choice and do you have a choice and that may be the number, right? Right. Yeah. Unless you can get to surgically pull small pieces out. But that’s what the business has to tolerate. You absolutely can. So, you can go in at that level and pull the pieces out and just restore, say, okay, this looks clean.

Let’s restore that. But understand we’re trying to restore an organization, a manufacturing organization that supports, all of the major automotive manufacturing, along with a lot of the industrial sector for bearings and they don’t want to wait three weeks, four weeks, five weeks until you get up out of them, however. So, we we started, the restoration process and we found a clean cut off for our data, some backups that were untouched, clean and started restoring those.

And of course, Active Directory and Commvault’s recovery, was not part of our stack at the time. So, you have to make a decision not just from a data perspective, but also from, your Active Directory perspective. Right. Do I trust this Active Directory? Because my data can look fine, but I don’t know who has compromised accounts in this.

So, we had to make that decision. We basically, stood up our primary domain controller at our data center, restored from back up to that AD and just started cleaning. So then that becomes restored in an isolated environment. Yep. And then just forced password changes for everyone across the board. Doesn’t matter if you changed it last week last month.

Get ready. You go ahead and get ready. I’m curious, did you also tighten password security policies at the same time? Are you enforcing our password policies? We’re we’re fairly robust. We have a 20-character minimum. So we we are we get a lot of pushbacks from that as well, right? Yeah. I mean, it’s like war and peace kind of push it.

And we have restrictions, right? You can’t do password 123. Password 123. You can’t put you know all of these common phrases. So 20-character password uppercase, lowercase numbers, letters, special characters. We had a pretty robust one of that. What was the challenge or what was the actual, weakest link in that chain was an old user account from a migration.

So, if you have organizations that are growing inorganically through acquisitions and you just kind of fold all of that in without doing the due diligence of reviewing those types of accounts and what access those accounts actually need, that was kind of what happened in our instances. It didn’t come through North America. There was, another account in a different region that was able that was already stood up with an old password requirement siphon, siphon.

Now, in this, teaching, this is the teaching moment. Yes. Or once. So, one of the many for sharing that. But since lots of folks go through M&A, right. Yeah. This is something that can unfortunately, you could squash it once. And if you don’t put a policy in place next M&A activity could happen again. You can have the policy in place.

We had a good policy for managing that. It’s you’ve got to give your resources the opportunity to review what’s necessary and make sure that anything that you’re pulling into your system meets the requirements in the policy that you have in place, and not some previous policy from a different organization. Makes sense, makes sense. So, this this happened. Is this what you were referring to?

This was sort of the core point that they got them through. And so, when you did the restore, is this how it has you say rinse and repeat it again. Was this the or was this the clean restore or is this where you found the anomaly again? No, this was the clean restore. So, we got the clean restore up.

And of course we have an incident response plan. We have a fairly comprehensive incident response plan that has a hierarchy for our restoration activities. And, you know, and it’s really a pyramid. Okay. You focus on restoring corporate access because you’ve got to maintain communication across your entire organization. You’ve got to be able to send the emails out.

You’ve got to be able to send, any certain files or documents, or you’ve got to get that out to the rest of the organization. Then we kind of break down and that’s a small group, right? That’s right. 20 or 30 or 50 people maybe then you break down to the next group is restoring production. Yeah. As a Japanese organization, a lot of our customers operate on a just in time inventory model, so they don’t keep days and weeks and months of inventory that supply those supply chain orders are coming almost in real time.

And to your point, we’ll stop that train. If you stop it for two days. You’re I mean, it’s hard to catch up. Yeah. And then of course, you’ve got liquidated damages. You’ve got, you know, contractual obligations that you’ve got to uphold. So, you want to get your production in our perspective. You want to get the production back up and running so we can continue to produce those parts.

That’s a larger group of people, you know, that may be several hundred people. The next group is going to be your, your warehousing and making sure you get those restored and your sales offices and, you know, the pyramid base and use base of users continues to grow as you’re doing this restoration.

So a solution such as your, the recovery solution, that Commvault to be able to restore that at whatever level you need to restore it. Right. You can do it manually by down to a specific domain controller. You can do a complete forest restoration, right? The whole thing without having to go through the challenges and obstacles and, and and difficulty that the Microsoft’s restoration guidelines tell you to go through.

And, you know, very large order is very large, but going through them is it’s a significant right. So, if you can set up a runbook, you know, in Commvault’s AD recovery solution, understand how that works, do some test restores from that. It puts you far ahead of the game. Well, first off, appreciate you being the client to do it.

Take it. I don’t even have to say anything. You’re doing that. You’re doing the work for me. But I appreciate that. And frankly, I’m curious. You get to that. You did not have that tool when this happened. No. So if you look back on this entire thing, beside the right tools for the job. What were the other key lessons sort of when you look back as you got distance of course. Now to that. Right. And congratulations on having well defined recovery. And resilience plans.

Communication out of band plans. You clearly had all your homework done there. And that’s far ahead of what a lot of folks are. But now when you look back, how comfortable do you feel and what cadence do you practice those restores or those AD restores to God forbid you prepared that that would never happen again?

Yeah. I think it’s would, would be very critical.

And Chris, you make a point. It’s not if it happens again, it’s when it happens again. A lot of other sessions here this week, in Nashville, have focused around resilience and recovery. Yeah. AI is becoming a much more challenging foe in all of this. You’re being kept saying, check this. Yeah, it’s very challenging. If there’s one piece of advice I could give to anyone that’s out there in that field, a plan is only as good as how you follow it.

We had a plan. Yeah. And then, of course, the hysteria. I wouldn’t say hysteria, you know, the nervousness and the unknown. Okay, awesome. I mean, these are, you know, you could practice it a ton of times, but when it happens for real, that’s what you’re getting a lot of different requests. At one time we had 27 number one priorities.

You can’t do that. You’ve got to have a plan. It’s got to make sense, and you’ve got to follow the plan regardless of how painful it may seem to be at that time. Right. Because you have to trust it, right? You have to trust the plan. And the people that’s in charge have to trust you. Yeah. To execute that plan.

So that that is probably the key. You can have all the tools in the world, right? You can have any type of solution out there from a backup perspective. You can have it from a recovery perspective. But if you don’t follow the plan that you’ve laid out to ensure that those steps are followed as precisely as possible, and every plan should have the ability to have a little flex into it.

But you can’t continue to change priorities and move forward because the heat of the moment, not in the heat of the moment. That’s why you put all the work into the practice, the rigor, the muscle memory. And you do have to just you have to trust what you’ve done in the investment. You’ve invested in people, you’ve invested in equipment, you’ve invested in technology.

Yeah. Well, I got to say, you know, I know you had a short time, but I think the way you told the story walked everybody through that. Again. We do not get to hear this that often from somebody who is sort of in the seat went through it. You’re very calm, individual. But I’m sure in the room it was, there’s a lot of tense moments.

There were a lot of tense moments and a lot of long hours. So, as I had mentioned, it was me and another gentleman, who I think very highly of that. We work through this process. So, it was 20 hours a day on site in the office doing restorations. Then we would leave, then we would be at home connected to VPN or direct connection, and we would look to see what’s being restored.

We’d have to keep monitoring what was being restored, because another one of the challenges, and I don’t want to expand on this too much, is is our data management. You know, the philosophy of keeping everything forever just doesn’t work in this day and age. So, when you’re trying to restore from a backup that’s 12TB in size over a network, right?

It slows the whole process down. Right? Right. And that may be a tough one to account for in the planning, because you’re not in a test mode. You’re not restoring necessarily, let’s say 12TB and your data is growing every day. Right? It’s growing every day. So, you know, some good stewardship of the data. Reviewing of the data is ultimately retention policies.

Billy, thank you for for sharing the story again. We don’t get that often from a firsthand account. And companies willing to share that and, for Commvault Chris as well. And Billy, thank you for being here.

Thank you.