Young users online: How technology can help build safer social platforms

The digital age is an essential part of young people today. With the Internet enabling connections with friends, discovering new interests, and facilitating learning/self-expression through digital platforms, younger generations rely on the Internet as the primary means of engaging in today’s world. While the amount of online participation from youth continues to grow, so, too, do the concerns regarding the safety, privacy, and responsible design of platforms that support youth.

AI-Driven Content Moderation

One of the most powerful advances in technology solutions employed to create safer platforms is through AI-powered content moderation. AI technology today enables systems to detect harmful content (e.g. cyberbullying, hate speech, and inappropriate material) in real-time across the Internet. When incorporated into a technology system, these AI-enhanced analytical software applications can analyze a user's text, photos, and videos and identify and/or take action to suppress inappropriate content prior to its becoming prevalent or widespread. AI technology is expected to be commonplace in 80% of major social networks by 2026 for providing large-scale monitoring of user content. While human moderators remain eager for better decision-making regarding nuances relative to user content, AI systems will continue to improve both the speed and efficiency of identifying users’ risky behaviors.

Age Verification & Protecting Digital Identities

Another critical component is the use of advanced age verification technologies. There are a number of platforms that utilize AI deriving facial age estimations; providing valid ID confirmations; and employing parental control software, thus providing not only for ensuring that youth are engaging with appropriate content and communities, but also in protecting the identity of youth online, all while providing parents and guardians with the tools they need to help keep youth safe online. Privacy-centric technologies will ensure that personal user information/data is not compromised during the age verification process.

Establishment of Enhanced Communication through Effective Filters

With advancements in technology, many digital platforms have created new ways to proactively mitigate the chance of experiencing harm from interpersonal interactions. Smart messaging filters will allow digital platforms to identify potentially abusive communications before delivering the message, which should encourage users to reflect on the messaging that they send. Digital platforms are increasingly providing "pause and rethink" opportunities when they identify potentially abusive communications. Research completed by a variety of organizations that promote digital safety suggests that these types of nudges can decrease the likelihood of people interacting in a toxic manner by 20%-30%, and encourage the promotion of more respectful communication in youth users of digital platforms.

24/7 Moderation and Community Engagement

As many digital platforms operate in a global environment, the need for continual moderation systems is becoming even more critical. With the assistance of artificial intelligence (AI), round-the-clock monitoring of all forms of activity on digital platforms allows for timely identification of harmful activity, suspicious accounts, and coordinated harassment activities. Additionally, the use of automated reporting systems allows users to easily report problematic behavior, which enables the communities themselves to assist in creating safer environments.

Creating a Safe Platform through Design

Safer social media platforms can be created through thoughtful design and creativity, as well as moderation. Examples of tools commonly found on social platforms include: customizable privacy settings, tools to allow for anonymous reporting, restrictions on users from sending messages, and allowing the user to control the amount of visibility to their online profile. As technology continues to evolve, the focus of developing safer environments for users is moving toward the proactive support of user safety (i.e., through AI and responsible design) rather than the reactive support of user safety through moderation.

(Dr. Kanishk Agrawal, Chief Technology Officer at Judge Group India)