Sections

Inside the development crunch pandemic – part one

Monday, 13th January 2014 07:56 GMT By Dave Cook

Is development crunch an evil born from poor management or is it a necessary facet of game production? VG247′s Dave Cook asks over 20 developers for their view, revealing stories of sacrifice, compromise and thoughts of suicide.

“I know a lot of people that will complain about working eight hours a day and call it crunch. Some people wear an occasional all-nighter like a badge of honour and then there’s people that haven’t been home in a week.”

Most of us have worked a period of overtime in our lives. Perhaps it was forced upon you without room for negotiation, or you felt compelled to help your boss and team out simply because you enjoy your work and want to see the company succeed. Sometimes employees are rewarded bonus pay or time off in lieu for their efforts, while others – typically salaried staff – are given no such compensation. It is simply expected of them to get the job done.

It’s no secret that game development, with all of its pre-planned marketing ‘beats,’ release schedules and launch days goes hand-in-hand with overtime. ‘Crunch’ is the term commonly used for the period at the end of a game’s development that sees employees cramming hard to make sure the title is ready to ship on time. Failure to meet that deadline can prove costly, especially in the blockbuster arena with multi-million dollar budgets and investor relations on the line.

For years, crunch time was scarcely addressed in public forums but it was there, disrupting the work-life balance of countless employees. All across the industry, parents were missing their children’s birthday. Their home life and relationships were suffering, yet no one was speaking out to expose the issue. It was the elephant in the room. That was until the partner of a concerned EA employee stepped forward and published a harrowing article that detailed gruelling working conditions at the company.

‘EA: The Human Story’ was published on December 14, 2004 by fantasy writer and GameWatch co-founder Erin Hoffman. Using the pseudonym ‘ea_spouse’ , Hoffman revealed a sadistic and bleak reality long hidden from public eyes. During his job interview with EA, the employee in question – later confirmed as Hoffman’s then-fiancé Leander Hasty – was asked if he minded working long hours. Because overtime was expected in the games industry, he accepted the job offer. Within months he was working twelve hours a day, six days a week. It was merely a taster of things to come.

The employee was offered no overtime pay or extra holiday allowance for his efforts, and the crunch period was regularly extended. Hoffman claimed that the company’s response to resistance was ‘If they don’t like it, they can work some place else.’ The internet caught wind of the blog and quickly, developers from other studios started stepping forward with similar tales of crunch horror and unreasonable working patterns.

Ultimately, Hoffman’s blog resulted in a run of class-action lawsuits against EA, and in 2007 the company issued claimants $14.9 million dollars in unpaid overtime. However, it was clear the issue was global and certainly not unique to EA. It was a pandemic.

But who is to blame? Is crunch the result of projects falling behind schedule due to the incompetence of high tier power-holders, or is it simply an extension of the deadlined nature of the business, with its countless embargoes, industry events and other dated milestones? These questions were asked following Hoffman’s expose and the conversation ran hot for a while, but ultimately it fell by the wayside once more.

That was, until Crytek – a mere month from launching Ryse: Son of Rome – posted a tweet that read, “By the time Ryse ships for Xbox One, we will have served the crunching team more than 11,500 dinners throughout development.” It was written as a light, possibly humorous factoid, but the studio’s followers weren’t laughing. Most were astonished that the studio would openly admit the scale of crunch conducted by its staff.

It suggested what we already knew; that development crunch time is still here, running people into the ground and forcing them to work unreasonable hours with no extra reward. But is this really the case, or is Crytek one member of a minority? What causes crunch and how does it impact workers? Can something be done? I wanted answers, so I posted an open call for developers to come forward anonymously and share with me their experiences. Within a day I had received almost 30 responses from willing participants.

Defining ‘crunch’

My interviewees came from countries around the world and teams of various sizes, but there was a stark consensus among the pack that crunch time is very much real and isn’t going away any time soon. These are all real stories from real people eager to get their message to you, the consumers, so that you can gain insight into how your games are made. I started by asking each respondent for their on definition of ‘crunch.’

One developer wrote, “Crunch can mean, but not all of – Voluntary: people wanting to do more off their own back. Enforced: workers must do more to meet deadlines but are not paid for it, and Overtime: workers get paid for extra work. There’s also crunch and then there’s CRUNCH. Doing the occasional late night is crunch to some people.

“I know a lot of people that will complain about working eight hours a day and call it crunch. Some people wear an occasional all-nighter like a badge of honour and then there’s people that haven’t been home in a week. And there’s people that think crunch means staying late playing games before you get free food and go home.”

The term seems malleable then, but the majority of responses poured real scorn on the notion of crunch and suggested that they had all fallen foul of unreasonable working terms, as opposed to the occasional late night. One interviewee at a triple-a studio revealed to me that he was once told by an angry colleague that crunch time is a small price to pay for the privilege of working in games development. It’s not hard to see that this isn’t necessarily compensation for having a shattered work-life balance.

“The four of us were locked into our office by our manager, saying that we were not allowed to leave our computers until our work was over, and that if we didn’t finish on time we would all lose our yearly bonus. At each meal time the manager opened the door and gave us each a McDonald’s bag, then left and locked us back in.”

Another worker agreed that opinions change on the matter, but added, “Some view it as an essential part of development where the game gets finished and that it brings the team together. I’m in the camp that believes it to be completely unnecessary, bad for the industry and its members, and should be avoided at all costs. Others acknowledge that crunch overall negatively affects the industry, but see it as a necessary evil and unavoidable. It seems like the industry is trending towards the ‘completely unnecessary and harmful’ camp, which is good to see.”

Unsurprisingly, another developer conceded that crunch can often be of your own making and in that case, is often easier to manage than having a mammoth task pushed on you without warning. “That’s a little easier to get on with,” they explained. “You know when you’re coming up to milestones (alpha, beta or gold) then it’s really every minute counts. I found the most intense periods to be around game conventions, especially if you’re showing new playable content. No one really complains, it’s like the idea of going home on time is a reward almost.”

For some workers, hitting those deadlines is a personal objective. After all, pouring hard work and passion into a years-long project, only to see if fail or release in poor health simply isn’t an option for one respondent. They told me, “I think it’s important for me to point that I’m working in Japan, which has a different working ethic than most western countries. That said, I worked at a studio for a year and had a very large share of crunch time there as well. I think I’ve come to accept it as part of the job. Most developers I know are very proud people. Being unable to deliver seriously sucks, and feels as a failure. Hence most of us put up with it to get the job done.”

It was noted by one employee that speaking on behalf of such a diverse industry in blanket terms can give an inaccurate depiction of crunch. After all, crunch for a 200-strong studio working on a blockbuster franchise will never be the same as two coders working on an iOS title out of a rented office. There was one constant they agreed upon, however.

“Not all studios operate the same way,” they cautioned, “and even within studios, not all teams operate the same way. That being said, I think there is a common attitude that crunch is just part of the job. It’s pervasive, and I think it’s dangerous. Acting like it’s just part of the job insinuates that it’s unavoidable, which is false. I’ve spoken up against it and I’ve done my best to avert it. Part of my job as a producer is to avert it. Being given the right channels to express problems with it is a lot more common than actually taking steps to avoid it, though, and I think the former is sadly more common than the latter.”

“Even outside of crunch, I think large parts of the games industry lack a proper respect for work-life balance; the attitude that if you’re only working eight hours a day, you aren’t really committed, is common. This doesn’t just affect the games industry, but I think we have it worse than a lot of other jobs. People feel privileged to be working in this industry, and sometimes that privilege translates into working too much, for a variety of reasons.

“People can be pressured by superiors, feel pressured by peers, or it can be entirely self-centered. If you’re the person working 10-12 hour days and you really feel like you’re pulling extra weight, it’s easy to feel prideful about that and be sceptical of the person who only works eight.”

It’s obvious that crunch is inevitable in many ways, but the origin of that last dash to release can vary, as can the attitudes of those encountering it. Given the regularity of crunch at studios in different countries, each bearing unique payrolls and work ethics, it’s not a leap to suggest that there are many factors at play when crunch becomes an unfortunate necessity. I then asked my interviewees for their own theories as to why crunch occurs, and was met with some rather harrowing replies.

Why does crunch happen?

It’s clear that crunch time is a thorny issue for many game developers, but it’s not entirely clear why it happens. When you think about it for a moment, yes, projects approaching release that still require work are most likely to hit harder periods of crunch, but what causes this eleventh-hour scramble? It’s easy to point fingers and accuse faceless executives and big entities like EA and Crytek without true rationale. Is it due to a time-management problem, lax attitudes at the project’s outset or something deeper and more complex?

“In the first company I worked for, we were about fifty programmers with ten managers above us,” a 15-year industry veteran told me. “The team of four people I was in was in charge of the online shop of the game, adding the products for sale, managing the special offers, programming the new payment methods, all that kind of thing. A bigger team was working on a second version of the game, with a brand new engine and this version didn’t have a release date, it hadn’t even been announced to the public yet. So there wasn’t any kind of pressure or anything, until one of the banks investing in the firm said ‘you’ve been working on a new version too long, release it next month or we stop giving you money’.”

“Game projects are gigantic. Details are bound to be overlooked and changed at the last minute. Which to be honest is fine, but at the end the client wants these changes in there as soon as possible. The project manager can’t say no, because they don’t want to lose the client.”

Crunch time set in, and once a version of the game was finalised and approved by the studio’s QA team, relief started to set in. That was, until someone in the upper echelons decided that the online store needed changing, and asked the coders to create a new one in 48-hours. By this point, the new store and updated version of the game had been announced to the title’s 10 million already-active players. The real crunch was only beginning, and got worse from there.

“The four of us were then locked into our second floor office by our manager, saying that we were not allowed to leave our computers until our work was over, and that if we didn’t finish on time we would all lose our yearly bonus (about €3,000 each). At meal time the manager opened the door and gave us each a McDonald’s bag (paid from our own salaries), then left and locked us back in. We all worked extremely hard during those 48 hours and managed to reprogram most of our features – all the ones that the players would see.

“We decided to leave our monitoring tools for later and took the risk of having payments running live without being able to check if it was working properly. We finally got out around 6am on the Friday morning, went to one of my colleagues flats to have a shower and were back to work at 9 o’clock, as requested by our manager.”

It’s a grim depiction of crunch caused by last-minute change, and one that could be pegged as rare. But as it happened, this developer was not alone. Other respondents agreed that changes to a game’s code can disrupt work-flow and shift patterns to an unfathomable degree, while producers were regularly singled out as central to the issue. Another worker told me, “Multiple avenues exist to reduce the impact of crunch, but studios aren’t taking advantage of them. Nearly every project I’ve been on, producers have not been aggressive enough to cut features that aren’t worth the effort.

“Standards aren’t high enough for code changes, so poor designs and hacks get checked in, even with peer reviews. Nearly every code base I’ve worked on has been filled with hacks to meet deadlines which don’t ever get fixed, and these hacks cause problems later in development (this is known as ‘technical debt’). Because fixing these hacks doesn’t provide a tangible benefit that can be seen in-game, they stick around, even in the shipped game. As an example, code for an E3 build often never gets removed. There are also tons of inefficiency in pipelines and tools. If these got fixed, team members could work faster and more efficiently, thereby reducing the need for crunch.”

It was also suggested that rising expectations from the paying public is also a catalyst for crunch, particularly in the triple-a scene which is constantly pushing production spend into the stratosphere. One developer explained, “You could lay the blame at publishers wanting the most for their money but then you can equally lay the blame at gamers wanting the most for their game as much as an artist wanting the most out of their art.

“Developers tend to get a lot of hate thrown at them for things that are entirely out of their control and, you know, management are developers too. Fact is, we’re making products that people buy and even the best planned projects fail for reasons too wide to go into here. As soon as you have a relationship with a customer where you need to deliver something in return for money, you are beholden to making that happen. What can we do? Lose the customer or work harder to deliver?”

It’s a good point, and goes hand-in-hand with all the Metacritic-chasing publishers have been doing since the aggregation site became popular. Back in 2012, reports detailed Bethesda’s promise to give Obsidian employees a bonus if Fallout: New Vegas hit a meta score of 85. It rested at 84 and no such bonus was rewarded, despite the years of work poured into the project. Did the studio crunch hard to get the game in a shippable, ’85 meta-score’ state before launch? It’s not beyond the realm of possibility.

“Budgets and release dates are often fixed so if there turns out to be more work in completing a game than expected, the only thing that can be done is find more time from somewhere or reduce scope. So, instead of ‘poor project management’ I blame ‘undisciplined leadership.’”

Another running theme throughout my interviews was that management and paying clients have little to no understanding of the development process at code level. What they might feel is a simple fix is regularly not the case. An engineer I spoke with claimed that all projects must be written ‘to spec,’ rather in modular code.

The latter is relatively easier to change, should the client request amendments down the line, but commonly, modular code isn’t allowed for various reasons. Engineers are warned against raising this and other issues to clients. “You don’t bite the hand that feeds,” they told me.

One engineer added, “So right there we have a frequently occurring situation. The engineer codes according to the spec and wants to reach deadlines so writes his code only according to spec. The engineer then becomes angry when the PM (project manager) tells the engineer: ‘Hey, we had a chat with the client and they want multiple objectives per quest’. The engineer will ask ‘Why didn’t they think about this before?’ and the project manager might be inclined to say ‘Why did you make a system that only allows one quest. Can’t you think outside of the box?’”

“And this always happens. It always does. Game projects are gigantic. Details are bound to be overlooked and changed at the last minute. Which to be honest is fine, but at the end the client wants these changes in there as soon as possible. The project manager can’t say no, because they don’t want to lose the client, and the engineers will say ‘fuck you’, but end up making the feature anyway because of pride issues and because they don’t want to lose their job.”

Many of us have – at some point – blamed our bosses for how the ship is run. It’s a natural gripe when you’re feeling over-worked or simply fed up with your vocation. It was a common issue shared among the developers I spoke with for this feature, and it certainly felt like more than a simple moan. I was told, “You often hear people blame ‘poor project management’ for crunch time but I think this misses the point. It’s worth looking up the project management triangle to get an understanding of what project management is here. In short: if you want to do more you either need more resources (i.e. people and money) or more time.

“Budgets and release dates are often fixed so if there turns out to be more work in completing a game than expected, the only thing that can be done is find more time from somewhere (i.e. crunch) or reduce scope (i.e. cutting features – which generally makes the game worse so people don’t want to do it). So, instead of ‘poor project management’ I blame ‘undisciplined leadership’. The people in charge of a project need to be able to say no to a new idea or feature or agree to cut scope before they expect the development team to cover the cost of the features for them.”

It seems then, that management is bowing to the pressures of the client or publisher, which then trickles down to the creative teams, forcing more hours and mounting demands on their already-heaped plates. Is this now the nature of the industry – with its multi-million dollar projects and increasingly corporate ways? Can something be done about this seemingly entrenched culture? What is the true human cost of crunch time?

Click here to check out part two of this feature and find out what my interviewees suggested.

Latest

9 Comments

  1. sh4dow

    “I know a lot of people that will complain about working eight hours a day and call it crunch.”

    As usual, I would like to initially point out that I work in the film, not the games industry.
    However, based on the things people have told me and that I’ve read, it seems like they are very, very similar.
    And based on that, it sounds to me like you’ve interviewed one of them macho employees of the creative industry there. With the proper “rah-rah, don’t be faggots, we can do it!!”-mentality.

    Aside from the fact that I’ve NEVER come across anyone complaining about crunch while working 8 hours a day, this developer may or may not have left out a minor detail. Namely that those people may be working 6 or even 7 days a week. Meaning 48 or 56 hours instead of 40.
    Of course, those are probably just wusses in his eyes. Because anything under 70 is for lazy bums.

    That said, I do see people watching videos while they work (and frequently not while they have legitimate downtimes because they wait for renderings or whatever) or who are very liberal with the breaks they take. And then claim to have worked 10-11 hours, when it was really just more like 8.
    And on the one hand, it bothers me a lot, as I’ve grown up with a strict belief in the duty of an employee to always give 100%. Because that’s what you get paid for.
    But you know what? There are also the duties of an employer. And since employers in these industries have mostly been acting like assholes, I can’t say that I blame people much. For one thing, they know that if they consistently leave on time, they soon might not find another job in their city. And so they find ways to deal with the excessive overtime (because it never is just an hour or two here or there). Especially when it’s unpaid overtime. And while I’ve been lucky myself so far, I am constantly worried about my future in this insane industry, as I not only see many, many people bowing to pressure (while I don’t tend to) but also quite a few who break and exit the field for good.

    #1 7 months ago
  2. Docker Al

    Excellent, interesting article. Well done Dave.

    #2 7 months ago
  3. Darkfield

    Excellent read, looking forward the the next part.

    Although if I can make a suggestion Dave, these kinda articles are best for weekends where there are next to nothing to read on VG247 and although it’d be though to miss it among the Monday morning news storm, still it’d be better if we have something to read Sundays too.

    Thank you though, it is a great piece.

    #3 7 months ago
  4. bradk825

    Very interesting read. It does make me wonder what can be done about crunch. I suspect that as in other deeply entrenched practices, it may take drastic steps to correct.

    Does the class-action suit against EA need to happen to other publishers, investors and studios? After how many law suits will they decide that crunch, or at least some versions of it, are now to be considered legal exposure as opposed to necessity?

    I know from experience that legal exposure changes the view of corporations. Not because I’ve been involved in anything of the sort (thank goodness) but because I have felt the changes it brings. The giant book of rules you must follow for your organization to keep non-profit status in Canada used to be a pamplet. The rules you had to follow to have liability insurance is the same way. If you fail to follow these rules, or appear to have failed to follow them, your insurance company will drop you like a bad habit.

    So if 6 or 7 more big publishers were to be hit by class-action suits similar to EA, would that change their minds? These people don’t understand people, they understand numbers. They don’t think of the human cost of their expectations, they think about the financial cost of those expectations not being met.

    #4 7 months ago
  5. Zero@Dawn

    Excellent article, my only real question is why does nobody record or keep records when this happens? Especially the situation mentioned where employees were locked in a room for two days, I mean that company could be sued monetarily or even criminally in a case like that. I mean look what came from the EA Spouse Blog, a lot of the claimants got their unpaid overtime. Maybe it’s just because I’m an American but there are laws to protect against things like that here, and no amount of non-disclosures or threating can stop you from reporting if something is going on.

    #5 7 months ago
  6. Penney R. Tucker

    I’ve got my FIRST check total of $1681for one week, pretty cool. working from home saves income in several ways.I love that. I’ve recently started taking the steps to build my freelance Job career so that I can work from home. here is i started….. http://WWW.TeC80.COⅯ

    #6 7 months ago
  7. UKVampire

    Amazing article Dave happily retweeted both part 1 and 2

    #7 7 months ago
  8. Dave Cook

    @7 Thank you :)

    #8 7 months ago
  9. YoungZer0

    @4: Better upper management.

    #9 7 months ago

Comments are now closed on this article.