Facebook Prepares for the Metaverse, and for the Data It Will Harvest
After reviewing hundreds of patent applications in the United States, the Financial Times found the answer to the question of Facebook's, now Meta's, grand plan for the metaverse, the futuristic world it is planning for all of us. The answer should not be surprising: the company has registered multiple patents for different technologies that track, collect and analyze biometric data, from eye and mouth movements to all the muscles of the face and body. All this is intended to mimic users' facial features and animate them as realistically as possible inside the virtual reality Facebook would like to create, the metaverse. It is clear that this list of patents does not represent mature technologies, and it may take many years before they find their way into commercial distribution, but even now they point to Facebook's intentions in that virtual space, as well as basic ideas about how it imagines achieving these goals.
Anyone who has used VR headsets, or wandered through virtual spaces that are very, very early versions of the metaverse, can see that there is a graphics problem, without a doubt. There are problems of space, distance and depth, there are problems between points of contact, things sometimes look strange and unnaturally artificial, the so-called Uncanny valley. Overcoming these oddities, especially when it comes to animating the avatars, which will represent users, is a task that will determine the fate of the metaverse, because if virtual spaces are not pleasant enough to stay in and do not provide an experience that can at least replace the real one, not to mention surpass it, people simply will not use them. That is why these technological developments are essential to the success of this imagined future.
To achieve these things, Facebook is working in the expected direction. Among the patents detailed by the Financial Times are a facial-expression tracking system using a head-mounted device, which then takes that information and adapts it to media content. Another patent is a system of wearable magnetic sensors for tracking body movement, and another generates 3D avatars based on a user's photos, using a tool Facebook calls a "skin replicator." The patents found by the Financial Times also include ones related to monetizing that metaverse, including a patent concerning the display of personalized ads in virtual reality based on age, gender and interaction with the virtual space itself, and another patent intended to allow third parties to sponsor the display of certain objects in the space. Facebook explained from the outset that its new focus on the metaverse, in which it plans to invest a continuing $10 billion, will always include advertisements. One cannot, of course, blame it. Although it is branded as a social networking company, in the end its business has always been about collecting as much personal information as possible about its users, deriving insights from it and selling them to anyone willing to pay.
1. A business model that already incentivizes predatory behavior
This business model is controversial even today because it is known to incentivize a range of predatory behaviors toward users. Among other things, Facebook's algorithm writers are constantly seeking to build systems that flood users with inflammatory and manipulative information, so they remain connected and active. In recent years these systems have produced a series of negative social phenomena such as the spread of fake news, as well as a tendency toward automated virality of content related to violence and hate speech. Now Facebook wants to take this model, which is already controversial, and combine it with additional parameters such as biometric data, facial expressions, eye movement, (simulated) physical activity in virtual space and interaction with the world itself.
It is hard to overstate how Facebook is the least attractive player to carve the path of innovation toward the metaverse. The company is established and skilled, but suffers from very low trust among its users and is repeatedly punished by regulators for privacy violations and anti-competitive conduct. Is this the company we would want to begin collecting biometric information on hundreds of millions of users as well? Would we want every pore in our bodies, every movement of hair, every micro-movement to be recorded and cataloged? Would we want Facebook to take these and try to derive further insights through access to the body? Where are our eyes drawn, what paths do we choose to walk, what colors attract our attention, how long do we linger at social events, how do we talk, exchange ideas, whether we are dominant or shy, whether we are happy or sad, sociable or solitary, how much emphasis we place on our own appearance or that of others, what our weaknesses or strengths are, whether we are right-handed or left-handed.
The issue is not only the depth of the manipulation of emotion and wallet that Facebook is likely to exert on us with this new data reservoir in a new space, but also handing over biometric data to it. Beyond using such data to identify a single person, such as through facial, hand or iris recognition, today these data are also used to assess a range of traits. For example, there are now artificial intelligence systems that claim they can distinguish among a range of facial expressions or tone of voice and identify precisely the specific emotion they describe. But there is significant scientific criticism of the ability to reduce human expressions to simplistic schemas, not to mention trying to predict future behavior through them.
2. When the algorithm controls the users
If Facebook succeeds in developing products of this kind, the issue will no longer be ads, and placing one object in space to expose one user or another to it. The issue will be influencing behavior based on movements and biological processes that are not under user control. In some ways, this would amount to directly undermining freedom of choice. Steering a person whom the system identifies as happy and easily influenced down one virtual path, and a person it thinks is sad or introverted down another. As long as regulators around the world do not start catching up, users' actions will be guided by algorithms.
Today, tools related to digital identities are governed by different laws such as consumer protection, data protection and privacy, but these laws do not know how to deal with complex digital identities of this kind, ones that combine advanced new technologies such as artificial intelligence, physical and behavioral biometrics that identify a person based on their voice or other physical and behavioral patterns. This is despite the fact that even today it is known that voice recognition methods, as well as emotion recognition, suffer from racial and gender biases. The regulatory gaps are enormous. Turning us into machine-readable humans should not happen without any public debate about the limits of these efforts, especially when they are carried out by companies driven by economic incentives that may threaten or conflict with individuals' fundamental rights. Otherwise, there is a risk of further erosion of users' anonymity and autonomy.
The demonstration of Facebook's metaverse. The ability to overcome the limitations will determine its fate
