By: Alexandra Newe 

Does the process of de-aging and deep-faking celebrity stars in films constitute ethical and/or psychological ramifications?

In an industry that thrives off of youthfulness and stardom, technological advancements concerning the de-aging process and deep-faking have raised a considerable amount of controversy. From films such as Gemini Man (2019) to Rogue One (2016), de-aging and deep faking have not only been used to make actors look younger but also to bring them back from the dead. It goes without saying that the technology used to render these processes is nothing less than a marvel, and this paper will cover the logistics involved. However, it will also explore the ethics of bringing beloved stars back from the grave through academics such as Katheleen Looke and Lisa Bode. This paper will also contain a segment investigating this technology’s psychological effects through the implementation of Masahiro Mori’s theory of the uncanny valley. It is through these modes that the investigation into the ever-changing presentation of the celebrity face on the screen will be examined. 

The process of digital mapping and manipulation of the face can be traced back to the early years of cinema. Facial modification has been an integral part of filmmaking since the early twentieth century. The term refers to using prosthetic makeup or other techniques to alter an actor’s appearance to create a specific effect or character. Filmmaker, Karl Struss was one of the founding pioneers of facial transformation. Struss worked primarily as a cinematographer in Hollywood during the 1930s. He was known for his use of both light and shadow to transform the body and face on the screen. He worked closely with makeup artists and special effects technicians to create a variety of transformations for his actors. It was through his use of makeup and lighting that his actors could change subtly and dramatically on camera (Artist, et. al). A prime example of this is in his 1931 film, Dr. Jekyll and Mr. Hyde. In this film, Struss applied red and green makeup to the doctor’s face, and through the use of red and green lighting, transformed the character on screen. This was an extremely effective technique for the live transformation of the face, so long as the film was shot in black and white. Regardless, Struss’s techniques for facial modification paved the way for the digital face manipulation we are used to seeing in modern films.

Struss’s techniques would be used for the next half-century in cinema as the predominant form of facial modification. But with the onset of the 1970s, technology improved and the beginnings of computer programming brought along improvements in cinematic transformation. Dr. Paul Ekman is an American psychologist and former professor at UCSF. He created a system called the Facial Action Coding System. This system was and still is used today to objectively measure facial expressions of emotions. It was developed as a systematic method for identifying, describing, and measuring facial movements. These movements then correspond to specific emotions. FACS is built on the foundation that all facial expressions are made up of specific combinations of individual muscle movements, called “action units.” There are 43 action units that represent and correspond to a specific muscle group in the face. By analyzing the combination of action units present in a given facial expression, FACS provides a detailed description of the emotion being expressed. Facial Action Coding System is used mainly in psychology and neuroscience, but it is also used in computer science. The study of these expressions can then be transferred into a computer algorithm which is then able to recognize facial expressions in real-time. 

As cinema transitioned into the digital age, facial mapping through computer technology grew more advanced through the addition of micro-expressions and detail. MOVA contour was developed in 2004 by MOVA, a computer graphics company. MOVA contour was a massive step forward in facial capture technology (Perlman). It allowed for the creation of highly realistic and detailed 3D digital representations of human faces. The technology itself uses a system of cameras and specialized software to capture high-resolution images of an actor’s face from multiple angles. The software uses algorithms to analyze and map the movement of the actor’s facial muscles onto a 3D model of their face, which creates a highly detailed and realistic digital representation. MOVA contour’s ability to capture wrinkles, micro-expressions, and other nuances makes it an ideal technology for films as well as video games. Films such as “The Social Network” (2010) and “The Curious Case of Benjamin Button” (2008) have used MOVA Contour to age and de-age actors digitally. 

As this paper examines de-aging in cinema, “The Curious Case of Benjamin Button” is a prime example, as the film centers around Benjamin, who ages backward over the course of his life. Based off of the 1922 short story by F. Scott Fitzgerald, “Benjamin Button” was turned into a filmic adaptation in 2008 by director David Fincher. The filmmakers used a combination of prosthetics and visual effects. Taking inspiration from early filmmakers such as Struss, the team took Brad Pitt and outfitted him with makeup and prosthetics when he needed to play Benjamin as an elderly man. However, when Pitt needed to play the character as a younger man, MOVA contour was utilized, along with motion capture, to create a more realistic and believable performance. “Face Replacement” was the term used to describe the technique involved in the process of de-aging Pitt. To start, the team captured a high-resolution scan of Pitt’s face and then mapped it onto a 3D model. The 3D model was then sculpted digitally to create a younger version of Pitt’s face. This model would then be blended with the live-action footage that was shot of the actor. The motion capture, however, was used to create realistic movements on screen. The motion capture data combined with the facial mapping data from MOVA contour was used to then animate Benjamin Button digitally. 

The ever-increasing obsession with keeping actors young ties into Hollywood culture as a whole. With A-list actors such as Brad Pitt, the nature of celebrity culture intertwines with that of digital stardom. Former communications professor in New Zealand, Barry King explores this idea in his article in Celebrity Studies. King argues that the digital age has changed the concept of celebrity stardom. In his article he states that “the actor’s body, on stage or screen, is both a producer and a bearer of signs, [in that] the actor manipulates behaviors that are indices of him- or herself in order to produce signs that resemble (as icons) the indices of his or her own behavior as properties of a fictional character” (King, 252). From this, the celebrity becomes known for these behavioral performances and imprints their character onto the film character. This only becomes heightened through digital technologies like social media and video, as they have fundamentally changed the way in which audiences interact with celebrities. Digital stardom becomes characterized by a sense of immediacy and accessibility. Will Smith provides us with an example of a celebrity that was able to transgress from the celebrity before social media to what he has become today. Smith has done everything from starring as the fresh prince of Belair to a hip-hop career in music to an A-list actor in Hollywood Blockbusters. Playing off of his known fame throughout the latter three decades, he was a perfect casting for the 2019 film “Gemini Man”. The production team needed to create a younger version of Will Smith in this film. The film centers around an older Will Smith interacting on screen with a younger clone version of himself. The de-aged version of Smith was created using a combination of face replacement and motion capture technology, similar to Brad Pitt’s Benjamin Button. The use of de-aging in the film allows audiences to re-engage with Smith’s earlier career in the entertainment industry. Through the eyes of Barry King, the digitally de-aged version of Smith acts as an index that we then connect to older films starring Smith. However, the technology raises a number of concerns that make up a significant portion of King’s argument. This is the ethics involved in the prolongation of star employment. There is a potential for a negative impact on actors’ careers as well as their reputations. This being that the older actor can be taking the job that a younger up-and-coming actor would have normally taken. In the instance of “Gemini Man”, a few decades ago a young actor with a resemblance to Smith would have been cast. Perhaps even his son, Jayden Smith, could have played the clone. This brings to light a double-sided issue. On one end of the conversation, cinema has always been an art form that thrives off of technological advancements. Whether that be the invention of “talkies” or CG animals, cinema is built off of this innovation. The advent of de-aging in the film introduces a new and thrilling way of interacting with stars and their respective characters on screen. 

Director of “Gemini Man”, Ang Lee admits that the de-aged Will Smith is not perfect and in well-lit scenes looks “goofy”. But in his eyes, “Gemini Man” exists as a beginning of a new era. In an era during which Hollywood stars are dwindling, de-aging can bring them back to life (Li). In essence, it is a form of immortality, in which the household name then becomes a reusable image. This science fiction-sounding idea isn’t too far-fetched, as Barry King would argue likewise. King’s theory states that as stars like Will Smith and Brad Pitt age, they will rely increasingly more on these digital technologies to remain relevant and marketable (King, 258). 

Moving into even more complex variations of de-agin technology, the visual effects company, Industrial Light and Magic created a markerless facial capture technology. The system was designed specifically for Martin Scorsese’s 2019 film, The Irishman. This film tells the story of Frank Sheeran, a WWII veteran turned mobster. The Irishman is an epic spanning the course of 4 decades. Because of this, Scorsese worked closely with ILM to develop what they coined as FLUX. It was designed for de-aging as well as rejuvenating actors on screen. Much like MOVA contour, FLUX allows for the creation of highly realistic micro-detailed versions of actors’ faces. However, unlike the extremely intensive light-sealed room and hundreds of cameras involved in MOVA contour, ILM uses a much simpler technique. FLUX uses a combination of three high-resolution cameras, usually ARRI Alexa and RED, to capture the performance of the actor. What makes FLUX so impressive is that it doesn’t require any traditional animation. Though the process is quite complex, FLUX is capture the principal photography in both normal light as well as infrared. The captured performance would then be transferred onto a digital model made up of previous performances of the actor (Pederson). The de-aged version of the actor can then be manipulated in any way with the software automatically adjusting for lighting changes as well as facial expressions. FLUX was used primarily in The Irishman to de-age Robert De Niro, Joe Pesci, as well as Al Pacino. To get the most realistic-looking faces, the software was developed with an AI-based face finder, that collected thousands of screengrabs from the actor’s expansive performances throughout their careers. From these screengrabs, combined with FLUX technology and principal photography, The Irishman was able to believably de-age De Niro, Pesci, and Pacino and the visual effects team would go on to be nominated for an Academy award. Though the feat achieved is massive, the achievements do not go without instigating an important discussion. 

The last several sections on digital de-aging in cinema have gone in-depth into the technicalities involved in making these feats a reality. However, these aesthetic choices do raise a number of concerns among moviegoers and industry professionals. To start, we must look to Kathleen Loock, professor of American studies as well as media studies in the English Department at Leibniz University Hannover. In her article published in the Orbis Literarum, she delves into the realist aesthetics as well as the cultural impact that de-aging has upon cinema and humanity. On one end of the spectrum, Loock argues that de-aging provides an answer to certain narrative problems, as age gaps and time warps no longer require extensive work to make a possibility. The de-aging of Will Smith in the previously mentioned Gemini Man supports the suspension of disbelief that there are two versions of the famous actor from two eras of his career. De-aging technology can be an extension of the realism involved in cinema, as the tech creates the capability to create a more authentic-looking portrayal of the past. But, a significant number of issues have been raised since this tech has made its breakout. Not only do people fear that de-aging puts younger actors out of work, but it also fails to create normal roles for older stars, particularly women. Much of the technology used has been programmed primarily for men, and male-dominated films. Loock claims that there is ageism embedded in the industry that only becomes more prevalent through the repeated reliance on de-aging in cinema; the Hollywood obsession with youth (Loock, 216). De-aging these stars can create a false sense of identity and a fixation on youthfulness. 

While the majority of this case study has been examining de-aging in cinema, there is a supernatural facet of this technology that has not been discussed yet. Bringing back Hollywood stars from the dead is something straight out of science fiction, but it is precisely what has been happening in cinema over the last few years. Starting famously with Peter Cushing, who passed away in 1994, he went on to star in the 2016 film, “Rogue One”. Industrial Light and Magic brought Cushing’s character to life using CGI and motion capture. Actor Guy Henry performed the role of Grand Moff Tarkin in the principal photography, and a digital likeness of Peter Cushing then meshed onto the performance (Dolloff). The use of this technology brings to light an ethical issue. On one hand, the re-animation of these cherished actors can be seen as a way to honor the legacy of the deceased actor, and perhaps allow them to continue to entertain audiences for generations. But on the other hand, the process raises important questions about consent as well as the right of the deceased actors to control their own image and likeness. For example, actor Robin Williams sold his rights to a non-profit charity in hopes that it would never be used commercially post-mortem. But certain copyright laws allow for the use of actors after a given number of years. This is also true for the familial estate of James Dean. The rights to Dean’s image were sold to the production house Magic City Films back in 2019. James Dean, who died in a 1955 car accident is set to star in a Vietnam war film over 6 decades after his death. As the rights of deceased actors often lie with their family’s estate, it is up to them as to whether or not film companies can buy them and use them commercially. Although this process may be viewed as exploitative and disrespectful to the memory of the actor, it is a very real part of the industry. Much like Barry King’s research on celebrity icons, these stars represent an image or a brand. Although there was once a living person behind the icon, after death, the brand is subject to becoming corporatized unless it is protected (King, 257). Though this may sound Orwellian, it is important for filmmakers as well as studios to consider the ethical implications of using technology to bring Hollywood icons back from the dead. 

While we are on the subject of science fiction, it is time to address the elephant in the room. Perhaps one of the largest issues facing digitally rendered humans is something referred to as the uncanny valley. Starting with the term uncanny, this term was coined by Sigmund Freud. The uncanny is the feeling of fright or discomfort due to a familiar place or feeling, no longer feeling familiar (Freud, 125). In essence, the uncanny describes something familiar but strange. Freud theorizes that the uncanny comes from our set of primitive beliefs that challenge what we believe is real and not. This brings us into the uncanny valley. Masahiro Mori, a Japanese roboticist in the 1970s, coined this term. The uncanny valley refers to the phenomenon where the closer an artificial entity comes to being indistinguishable from non-humans, the more cute it appears. However, there is a dip in the graph between what we recognize as human-like, where the entity can be seen as unsettling or even repulsive. In other words, there reaches a point where the entity looks almost human, but there are still certain subtle differences that create the feeling of discomfort in the observer; this is the uncanny valley. Mori’s theory suggests that there is a delicate balance between creating life-like beings that are realistic enough to be engaging, but not so realistic that they become unnerving. When applying this theory to things like cinema, CG and motion capture begin to delve into the valley. Take Peter Cushing’s reanimation as Grand Moff Tarkin in “Rogue One”. Though the technology used to create the character is impressive, the attempt to create a digital version of Cushing became so life-like that critics like Rich Haridy argue that our emotional response becomes revulsion (Haridy). While Tarkin appears realistic for a moment on screen, upon further inspection, the small details such as the eyes and mouth movements are not perfectly realistic. The confrontation with death perhaps, is another facet of this phenomenon that makes Cushing’s appearance so uncanny. Because we are interacting with a character that appeared as they did 40 years ago in modern cinema, the knowledge of this fact increases the discomfort felt by the performance. Additionally, the added knowledge that the actor you see on screen is not played by the physical actor makes us hyper-aware of detail, and we unnervingly search for hyper-realism within the pixels. Haridy explains that the eerieness comes from both the appearance of the character, as well as the knowledge of the technology being used. 

This concept of the uncanny valley and its relationship to cinema pushes this paper into the psychological portion of the discussion. There is a significant amount to unpack when discussing the uncanniness of deep-faking and de-aging celebrities in film. People are very responsive to digital anomalies in animated human characters, particularly within the face. This dates back to primitive instincts where the human brain needed to identify faces. We have a nearly photographic memory when it comes to facial recognition, and thus our ability to discern between human faces and non-human is incredibly good. As far as this goes for the on-screen body, there is much more room for error in the digitally rendered body than there is for the digitally rendered face (Hodgins 12). The cognitive dissonance that we experience when we enter the uncanny valley adds to the sense that we are being presented with a reality that is not accurate. 

When discussing the uncanny valley, de-aging, and CG humans in films, motion capture becomes quite an interesting addition to this topic. The 2004 film, “The Polar Express” by Robert Zemeckis used motion capture technology to animate Tom Hanks into five different characters spanning ages 9 to 90. Hanks was 50 at the time of filming, and thus he moved and interacted with things like a 50-year-old would. Nonetheless, he was de-aged digitally to play the 9-year-old character, Chris. In order to use this performance capture data, the animators of the film needed to do “motion retargeting” in which the same animations can be recycled for separate characters. Due to the technological limitations at the time of the film, the end result provided audiences with a general unease. This was due to the fact that the technology lost many of the nuances and facial expressions that Tom Hanks gave in his performance. The end result is a film with stiff characters that seem to appear “lifeless” (Mahler).  

With this in mind, it is worth re-examining Ang Lee’s “Gemini Man” from the perspective of the uncanny valley. Film critic for the New York Times, Peter Sudman, argues that the computer-generated young smith is not as convincing. He refers to the digitalized performance as a ‘digital costume’ for Smith. He rips into the performance by pointing out that, “ there’s a slightly unnatural quality to the character’s brow and eyes, to the way his lips don’t quite seem to be in sync with his words; he’s more pixel than a person, an animated presence rather than a flesh-and-blood actor” (Suderman, 1). Part of this may be due to the creative choice to shoot the film in 120 frames per second, making the entire film feel surreal. However, it does go without saying that the animation of Young Smith is just not realistic enough to be fully convincing, particularly in high-exposure scenes. The uncanniness of the performance may well be attributed to the fact that the real Will Smith is sharing the same screen as the digital version. And we as audience members are fully aware of this, so our attention to detail becomes heightened. Though “Gemini Man” is far more realistic in its depiction of real-life people than “Polar Express”, the technology is not yet advanced enough to bridge the gap between real actors and the digital version. Barry King’s view on this stems from his explanation that cinema “imagines, but does not produce actual beings [and] represents them through embodiment” (King, 253). And thus, digital cinema is the most advanced expression of this practice. “Gemini Man” may just be the cornerstone of cyber-stars. Though the practice costs millions of dollars at this time, the more advancements we attain, the cheaper it will be to incorporate digital actors on screen. The prospect of replacing expensive actors with virtually free digital ones is practically the holy grail for production houses looking to lower their budgets. But we are still stuck in the uncanny valley, and this potential is still a fantasy, for now. 

In summary, the uncanny valley is a byproduct of our progress in technology. The closer we get to fully recreating ourselves, the more we pay attention to the imperfections. But the expression of this phenomenon in cinema points to a larger development. The technology used to de-age and deep fake celebrity stars in film can be and has been used for nefarious purposes. As this technology has become more open-sourced, so too has the illegal use of synthesizing video and images. Unfortunately, women are at the brunt of this situation. The Artificial Intelligence firm Deeptrace produced a study of over 15,000 deep fake videos in September 2019. Of the 15,000 videos, 96% were of pornographic content. Not only this, but 99% of the videos were of female celebrities (Sample). In essence, deepfaking technology is being weaponized against women in the form of revenge porn. This creates a whole other aspect of moral dilemmas involving the technology at hand. Even beyond porn, deep fake and cyber-animation is used for satire and false news, which is a whole other area of issue. I myself have done research on the use of technological advancements to promote conspiratorial thinking. In a case study on falsely generated content, it was deduced that “we look to the media for guidance and for awareness,, and with today’s collapse of transparent journalism, conspiracy is thriving. Where we once used to have a seemingly high standard of trust in journalistic integrity has gone out the window for the post-Trump era of fake news and paranoid politics” (Newe). Though this may seem like a stretch at first glance, the same technology used to deep fake celebrities in films is used as a weapon in the political arena. In essence, there needs to be a considerable amount of care drawn to this new and impending future where technology like this becomes available to everyone. There are countless psychological ramifications due to the misuse of deepfake material such as defamation, and if we are not careful, the familiar will become horrifying. 

All of this sounds quite sinister, and though a part of it is, there is still room for expressive creativity through the use of deep fake and de-aging tech. Particularly for cinema, reanimation can be a way to remember and honor a beloved actor’s legacy. It can also provide older actors with a longer career in life. However, we must scale the valley of uncanniness before we are able to fully bring these stars back from the dead, at least in a convincing performance. The challenges the future holds for this technology is something that the film industry will have to confront sooner rather than later. And perhaps we may enter a world where the two forms coexist in harmony through legislation that protects people. Nevertheless, the ethical and psychological developments stand to spark a discussion about a world that is ever-changing. 

Bibliography 

Artlist, et al. “The History of Aging and De-Aging on Screen: Artlist.” Artist Blog, Artlist Co., 22 Sept. 2022, https://artlist.io/blog/the-history-of-aging-and-de-aging-on-screen/.

Dolloff, M. (2017, February 21). A Look Behind the Tarkin VFX in Star Wars: Rogue One. ScreenRant; Screenrant. https://screenrant.com/star-wars-rogue-one-tarkin-ilm-peter-cushing-video/

Haridy, R. (2016, December 19). Star Wars: Rogue One and Hollywood’s Trip through the uncanny valley. New Atlas. https://newatlas.com/star-wars-rogue-one-uncanny-valley-hollywood/47008/

Hodgins, Jessica. Sophie Jörg, Carol O’Sullivan, Sang Il Park, and Moshe Mahler. 2010. The saliency of anomalies in animated human characters. ACM Trans. Appl. Percept. 7, 4, Article 22 (July 2010), 14 pages. https://doi.org/10.1145/1823738.1823740

Li, S. (2019, December 6). “The Irishman,” “Gemini Man,” and the Rise of De-aging. The Atlantic; The Atlantic. https://www.theatlantic.com/entertainment/archive/2019/12/irishman-gemini-man-and-rise-de-ag

ing/603130/

Loock, K. (2021). On the realist aesthetics of digital de‐aging in contemporary Hollywood cinema. Orbis Litterarum, 76(4), 214–225. https://doi.org/10.1111/oli.12302

Mahler, M. (2019, November 19). When de-aging De Niro and Pacino, “Irishman” animators tried to avoid the pitfalls of the past. The Conversation. https://theconversation.com/when-de-aging-de-niro-and-pacino-irishman-animators-tried-to-avoid-pitfalls-of-the-past-126559

Newe, A. (2021, March 15). The Golden Age of Conspiracy. Alexandra Brontë Newe. https://alexnewe.com/2021/03/15/the-golden-age-of-conspiracy/

Perlman, S. (2004). Mova: About Us: Overview. Www.mova.com; Rearden Companies. http://www.mova.com/aboutus.php

Pederson, L. (2020, August 12). Stories | RenderMan at ILM | The Irishman. Renderman.pixar.com; Renderman at ILM. https://renderman.pixar.com/stories/renderman-at-ilm–the-irishman

Sample, I. (2020, January 13). What are deep fakes – and how can you spot them? The Guardian. https://www.theguardian.com/technology/2020/jan/13/what-are-deepfakes-and-how-can-you-spot-them

‌

Sigmund Freud, & Mclintock, D. (2003). The Uncanny (pp. 121–162). Penguin.

Suderman, P. (2019, October 11). Gemini Man Is an Extended Visit to the Uncanny Valley. Reason.com. https://reason.com/2019/10/11/gemini-man-is-an-extended-visit-to-the-uncanny-valley/